February  2007  VoL.  H  No.  1 


A  PUBLICATION  OF  THE  DEFENSE  ACQUISITION  UNIVERSITY 


44 


Problem  Solving 
Collaboration  and  Teamwork  - 
The  Key  to  Successful  Warfighter  Contract  Support 
Lt  Col  (Sel)  Gilberto  Rosario,  USAF 
Joseph  Vernon 
William  Kleiner 
Matthew  Popham 

A  Business  Model  for  Defense  Acquisition 
under  the  Modular  Open  Systems  Approach 
Eugene  Gholz 

Decision  Support  for  Best  Practices 
Lessons  Learned  on  Bridging  the  Gap 
Between  Research  and  Applied  Practice 
Raimund  L.  Feldmann 
Forrest  Shull 
Michele  A.  Shaw 

Sense  and  Respond 

An  Emerging  DoD  Concept  for  National  Defense 
Russell  A.  Vacante 

Application  of  Structured 
Decision-Making  Tools  to  Defense  Acquisition 
LCDR  John  R.  Gensure,  USN  (Ret.) 

The  Challenges  and  Opportunities 
of  Implementing  Human  Systems  Integration 
into  the  Navy  Acquisition  Process 
James  A.  Pharmer 

Applying  Office  Leadership  Solutions 
to  Management  Recommendations 
in  Solving  Acquisition  Problems 
Martin  Cain 


PROBLEM  SOLVING  IN  THE 


Report  Documentation  Page 

Form  Approved 

OMB  No.  0704-0188 

Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 

VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 

1.  REPORT  DATE 

FEB  2007  2' REPORT  TYPE 

3.  DATES  COVERED 

00-00-2007  to  00-00-2007 

4.  TITLE  AND  SUBTITLE 

Defense  AR  Journal  (Volume  14,  Number  1,  February  2007) 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Defense  Acquisition  University, ATTN:  DAU  Press, 9820  Belvoir 

Road, Fort  Belvoir, VA, 22060-5565 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

10.  SPONSOR/MONITOR'S  ACRONYM(S) 

11.  SPONSOR/MONITOR'S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

14.  ABSTRACT 

15.  SUBJECT  TERMS 

16.  SECURITY  CLASSIFICATION  OF:  17.  LIMITATION  OF 

_ _ _  ABSTRACT 

18.  NUMBER  19a.  NAME  OF 

OF  PAGES  RESPONSIBLE  PERSON 

a.  REPORT  b.  ABSTRACT  c.  THIS  PAGE  Same  US 

unclassified  unclassified  unclassified  Report  (SAR) 

122 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


Frank  J.  Anderson,  Jr. 

President,  Defense  Acquisition  University 


Board  of  Review 


Victor  F.  Ciardello 

Director  of  Small  Business  Technology 
and  Industrial  Base 
Office  of  the  Secretary  of  Defense 

Domenico  C.  Cipicchio 

Acting  Director,  Defense  Procurement  and 
Acquisition  Policy 

Office  of  the  Under  Secretary  of  Defense 
(Acquisition,  Technology  and  Logistics) 

Dr.  J.  Ronald  Fox 

Professor  Business  Administration,  Emeritus 
Harvard  Business  School 

ADM  James  R.  Hogg,  U.S.  Navy  (Ret) 

Director,  Chief  of  Naval  Operations,  Strategic 
Studies  Croup 

Scottie  Knott 

Senior  Procurement  Executive  and  Deputy 
Logistics  Operations 
Defense  Logistics  Agency 


Ken  Krieg 

Under  Secretary  of  Defense 
(Acquisition,  Technology  and  Logistics) 

Dr.  Diane  R.  Murphy 

President  and  Chief  Executive  Officer 
Information  Technology  Management  Institute 

William  H.  Reed 

Director 

Defense  Contract  Audit  Agency 

Eleanor  Spector 

Vice  President  of  Contracts 
Lockheed  Martin  Corporation 

Kathryn  C.  Turner 

President  and  Chief  Executive  Officer 
Standard  Technology,  Inc. 


Editorial  Board 

Dr.  Paul  Alfieri 

Chairman  and  Executive  Editor 


Dr.  Richard  Donnelly 

The  George  Washington  University 

Randy  T.  Fowler 

Curriculum  Development  and  Support  Center, 
Defense  Acquisition  University 

Dr.  Ned  Kock 

Texas  A&M  International  University 

Dr.  Ira  Lewis 

Naval  Postgraduate  School 

Dr.  E.  Johnson  Marits 

The  George  Washington  University 


Dr.  Mark  Montroll 

Industrial  College  of  the  Armed  Forces 

Col  Mary  Kringer,  U.S.  Air  Force 

Commandant 

Defense  Acquisition  University 

Dr.  Keith  Snider 

Naval  Postgraduate  School 

Charles  E.  Tompkins  III,  Esq. 

Information  Resources  Management  College 


Eduard  H.  Boyd  Norene  L.  Fagan-Blanch  Dustin  Brown 

Director,  DAU  Press  Managing  Editor  Editorial  Assistant 


Bartlett  Communications  Paula  Croisetiere 

Technical  Editing,  Design,  and  Layout  Prepress  Production  Manager 


TSgt  James  Smith,  USAF,  and 
SPC  Kelly  Lowery,  USA 

Visual  Information  Specialists 


The  Defense  Acquisition  Review  Journal,  formerly  the  Acquisition  Review  Quarterly  journal,  is  published 
quarterly  by  the  Defense  Acquisition  University  (DAU)  Press.  Postage  is  paid  at  the  U.S.  Postal  facility,  Fort  Belvoir, 
VA,  and  at  additional  U.S.  Postal  facilities.  Postmaster,  send  address  changes  to:  Editor,  Defense  Acquisition 
Review  Journal,  DAU  Press,  9820  Belvoir  Road,  Suite  3,  Fort  Belvoir,  VA  22060-5565.  For  free  copies,  mail 
written  requests  with  an  original  signature  to  the  above  address  using  the  subscription  form  provided  in  this 
journal.  Articles  represent  the  views  of  the  authors  and  do  not  necessarily  reflect  the  opinion  of  the  DAU  or  the 
Department  of  Defense.  Some  photos  appearing  in  this  publication  may  be  digitally  enhanced.  ISSN  1 553-6408. 


The  Defense  Acquisition  Review  Journal  is  available  electronically  on  the  DAU  Home  Page  at 

http://www.dau.mil/pubs/arqtoc.asp 


TABLE  OF  CONTENTS 

FEBRUARY  2007 
VOL.  14,  NO.  1 


DEFENSE 


A  NOTE  FROM  THE  EXECUTIVE  EDITOR 


PROBLEM  SOLVING:  COLLABORATION  AND  TEAMWORK— 
THE  KEY  TO  SUCCESSFUL  WARFIGHTER  CONTRACT  SUPPORT 
Lt  Col  (Sel)  Gilberto  Rosario,  USAF,  Joseph  Vernon, 

William  Kleiner,  and  Matthew  Popham 

The  Defense  Contract  Management  Agency  (DCMA)  and 
the  Rolls-Royce  Corporation  discovered  that  through  close 
collaboration  and  teamwork,  day-to-day  business  issues 
that  previously  required  significant  time  and  resources  to  be 
resolved  instead  quickly  started  to  disappear  or  to  require  less 
effort  to  be  resolved.  This  article  discusses  the  underpinnings 
of  a  successful  government  and  contractor  collaboration  though 
the  discussion  of  DCMA’s  contract  management  services,  the 
contractor’s  environment,  technical  challenges  realized  within 
manufacturing  and  business  systems,  followed  by  a  look  at 
problem  solving  approaches  and  philosophies,  all  of  which  led 
to  significant  improvements  in  both  the  quality  and  on-time 
delivery  to  the  warfighters  at  a  fair  and  reasonable  price. 


21 

Eugene  Gh oh 

This  article  briefly  describes  a  business  model  that  companies 
may  apply  to  develop,  produce,  and  sell  avionics  to  the 
Department  of  Defense  under  a  Modular  Open  Systems 
Approach  (MOSA).  Recent  acquisition  reforms  have 


7  A  BUSINESS  MODEL  FOR  DEFENSE  ACQUISITION  UNDER 
THE  MODULAR  OPEN  SYSTEMS  APPROACH 


235 


249 


encouraged  the  use  of  MO  S  A,  and  the  defense  industry  will 
need  to  adapt  to  the  new  style  of  acquisition.  A  business 
model  summarizes  the  way  a  firm  earns  profits  sufficient  to 
remain  in  business:  it  describes  core  competencies,  principal 
activities,  cost  structure,  and  expected  revenue  stream.  The 
model  proposed  here  suggests  that  firms  can  succeed  under 
the  new  framework,  but  that  MOSA  entails  some  drawbacks 
for  both  industry  and  the  government  that  may  limit  its 
applicability  to  a  relatively  small  subset  of  programs. 
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components,  if  the  right  best  practices  are  applied,  they 
help  to  avoid  common  problems  and  improve  quality,  cost, 
or  both.  However,  finding  and  selecting  an  appropriate 
best  practice  is  not  always  an  easy  endeavor.  In  most  cases 
such  guidance,  based  on  sound  experience,  is  missing; 
often  the  best  practice  is  too  new,  still  under  study,  or  the 
existing  experiences  do  not  fit  the  user’s  context.  This  article 
reports  on  a  program  that  tries  to  bridge  the  gap  between 
rigorous  empirical  research  and  practical  needs  for  guiding 
practitioners  in  selecting  appropriate  best  practices. 
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community  suggests  that  its  meaning  and  necessity  are  not 
completely  understood.  To  help  make  sense  and  respond 
less  a  catchphrase  and  more  a  well  understood  concept, 
the  text  that  follows  will  address:  what  the  term  sense 
and  respond  means,  why  it  is  important  to  our  national 
security,  and  its  relationship  and  application  to  the  logistics 
community.  The  goals  of  this  article  are  to  provide  the 
reader  with  a  fundamental  understanding  of  the  sense- 
and-respond  concept  and  promote  greater  dialogue  among 
a  larger  group  of  interested  parties  on  this  concept. 
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making  tools  to  Defense  acquisition  problems  can  significantly 
assist  the  decision  maker  in  the  analysis  of  complex  decisions, 
particularly  those  involving  uncertainty,  risk,  and  multiple 
objectives.  Decision  analysis  and  operations  research  are 
structured  decision-making  tools  that  can  aid  the  decision 
maker  in  avoiding  biases,  documenting  decision  methodologies, 
and  making  group  decisions.  Overall,  the  systematic  application 
of  structured  decision-making  tools  can  significantly  increase 
a  decision  maker’s  insight  into  the  complex  decisions  that 
are  characteristic  of  the  Defense  Acquisition  System. 
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Over  the  last  decade,  the  Department  of  Defense  has  placed 
increased  emphasis  on  including  considerations  of  human 
capabilities  and  limitations  into  systems  engineering  and 
acquisition  processes.  The  purpose  of  this  article  is  to  provide 
an  overview  of  how  the  Navy  is  implementing  Human 
Systems  Integration  (HSI),  the  process  of  incorporating 
considerations,  characteristics,  capabilities,  and  limitations 
of  human  operators  and  maintainers  within  acquisition 
decision  making  at  a  level  commensurate  with  decisions 
regarding  hardware  and  software.  More  specifically,  this  article 
will  address  some  of  the  policy  initiatives,  organizational 
changes,  and  implementation  challenges  of  incorporating 
HSI  into  the  acquisition  life  cycle  to  insure  better  total 
system  performance  and  lower  total  ownership  cost. 
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Marlin  Cain 

Leadership  is  the  process  by  which  high-performance 
acquisition  takes  place.  Influences  on  management  as  a 
result  from  inspections  such  as  internal  auditing  can  pull 
attention  from  long-term  leadership  goals.  This  article 
researched  the  nonquantifiable  impacts  from  internal  auditing 
and  compared  them  to  effective  leadership  principles. 

Conclusions  indicated  that  management  could  make  better 
use  of  audit  recommendations  by  applying  leadership  to 
actions  in  solving  problems  and  implementing  change. 
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FROM  THE 
EXECUTIVE  EDITOR 

PROBLEM 
SOLVING 
IN  THE  21st 
CENTURY 

Welcome  to  the  Defense  Acquisition  Review  Journal  (ARJ)  theme  edition  on 
“Problem  Solving  in  the  21st  Century.”  Dramatic  advances  in  technology  and 
computing  power  make  higher-order  thinking  and  sound  reasoning  essential  elements 
of  problem  solving  in  the  modem  era.  These  skills  include  the  cognitive  processes  of 
analysis,  comparison,  inference  and  interpretation,  evaluation,  and  synthesis  applied 
to  a  wide  range  of  domains  and  problem-solving  contexts.  However,  the  basic  skills 
of  leadership,  personal  interaction,  collaboration,  and  teamwork  continue  to  provide 
effective  methods  for  reaching  solutions. 

On  behalf  of  the  Defense  Acquisition  University  (DAU)  Press,  the  Defense  ARJ 
managing  editor,  and  staff,  I  would  like  to  personally  thank  the  leadership  of  the  DAU 
Southern  Region,  Huntsville,  AL,  for  selecting  the  theme  and  sponsoring  this  edition 
of  the  journal.  Professor  David  Eiband  is  also  commended  for  his  tireless  efforts  and 
tenacity  while  leading  in  the  selection  process  and  coordination  of  this  issue  between 
DAU  South  and  DAU  Headquarters,  Fort  Belvoir,  VA. 

Thus,  the  edition  begins  with  an  article  entitled,  “Problem  Solving  in  Defense 
Contract  Management  Services:  Collaboration  and  Teamwork — The  Key  to 
Successful  Warfighter  Contract  Support,”  by  Lt  Col  (Sel)  Gilberto  Rosario,  USAF; 
Joseph  Vernon;  William  Kleiner;  and  Matthew  Popham.  In  this  article,  the  authors 
point  out  that  while  past  management  approaches  to  business  interactions  yielded 
less  than  optimal  results,  a  new  emphasis  in  collaboration  and  teamwork  between  the 
Rolls-Royce  Corporation  and  the  Defense  Contract  Management  Agency  (DCMA) 
quickly  produced  sizable  benefits  for  both  parties. 

Eugene  Gholz  describes  a  business  model  that  companies  could  apply  to  the 
development,  production,  and  sale  of  avionics  to  the  Department  of  Defense  (DoD) 
in  his  article,  “A  Business  Model  for  Defense  Acquisition  under  the  Modular  Open 
Systems  Approach.”  This  model  uses  a  Modular  Open  Systems  Approach  (MOSA), 
which  has  been  encouraged  through  acquisition  reform  initiatives  and  emphasizes  a 
thorough  understanding  and  application  of  core  competencies,  principal  activities, 
cost  structure,  and  an  expected  revenue  stream  to  enable  sufficient  profits  to  remain  in 
business  in  a  competitive  environment. 
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The  article,  “Decision  Support  for  Best  Practices:  Lessons  Learned  on  Bridging 
the  Gap  between  Research  and  Applied  Practice,”  by  Raimund  L.  Feldmann,  Forrest 
Shull,  and  Michele  A.  Shaw,  examines  the  process  of  selecting  and  implementing 
best  practices.  In  some  cases,  a  best  practice  may  be  only  appropriate  for  a  specific 
scenario  or  context,  and  in  other  cases,  a  best  practice  could  fit  a  new  or  emerging 
system  with  evolving  threats  and  tactics.  In  addition,  the  authors  describe  a  program 
that  tries  to  bridge  the  gap  between  rigorous  empirical  research  and  practical  needs 
for  guiding  practitioners  in  selecting  appropriate  best  practices. 

Russell  A.Vacante’s  article,  “Sense  and  Respond:  An  Emerging  DoD  Concept 
for  National  Defense,”  explores  the  importance  of  a  new  concept  emerging  from  the 
network-centric  environment  philosophy.  This  concept  is  relatively  new  within  much 
of  the  defense  community,  and  its  true  meaning  and  utility  may  not  yet  be  completely 
understood.  Dr.  Vacante  emphasizes  why  sense  and  respond  is  so  important  to 
national  security  in  the  21st  century  and  explains  its  relationship  to  the  logistics 
community. 

LCDR  John  R.  Gensure,  USN  (Ret.),  presents  an  overview  showing  how  the  use 
of  decision  analysis  and  operations  research  can  significantly  assist  decision  makers 
in  the  complex  world  of  defense  acquisition  in  his  paper  entitled,  “Application  of 
Structured  Decision-Making  Tools  to  Defense  Acquisition.”  These  types  of  tools 
enable  managers  to  trade  off  factors,  such  as  uncertainty,  risk,  and  multiple  objectives 
and  priorities  to  reach  optimal  decisions.  Systematic  application  of  structured 
decision-making  tools  can  significantly  increase  a  decision  maker’s  insight  into  the 
complex  scenarios  of  defense  acquisition. 

The  next  article,  “The  Challenges  and  Opportunities  of  Implementing  Human 
Systems  Integration  into  the  Navy  Acquisition  Process,”  by  James  A.  Pharmer, 
provides  a  look  at  how  the  Navy  is  implementing  Human  Systems  Integration 
(HSI) — the  process  of  incorporating  considerations  of  human  capabilities  and 
limitations  at  a  level  commensurate  with  decisions  regarding  hardware  and 
software.  The  author  also  addresses  policy  initiatives,  organizational  changes,  and 
implementation  challenges  while  incorporating  HSI  into  the  acquisition  life  cycle. 

Last  but  not  least,  “Applying  Office  Leadership  Solutions  to  Management 
Recommendations  in  Solving  Acquisition  Problems,”  by  Martin  Cain,  focuses  on 
a  study  of  leadership  as  a  primary  force  allowing  high-performance  acquisition 
to  materialize.  This  article  examines  influences  on  management  resulting  from 
inspections,  such  as  internal  auditing,  and  researched  the  nonquantifiable  impacts 
while  comparing  them  to  effective  leadership  principles.  Conclusions  emphasize  that 
appropriate  leadership  is  always  key  in  solving  problems  and  implementing  change. 

I  would  like  to  encourage  other  organizations  to  sponsor  editions  of  the  Defense 
ARJ.  If  you  have  an  idea  for  a  theme,  we  want  to  hear  about  it.  For  more  information 
on  sponsoring  an  edition  of  the  journal,  please  feel  free  to  send  inquiries  to  the 
managing  editor,  Defense  ARJ,  at  DefenseARJ@dau.mil. 

Dr.  Paul  Alfieri 

Executive  Editor,  Defense  ARJ 
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PROBLEM  SOLVING: 
COLLABORATION 
AND  TEAMWORK— 
THE  KEY  TO  SUCCESSFUL 
WARFIGHTER  CONTRACT 
SUPPORT 


Lt  Col  (Sel)  Gilberto  Rosario ,  USAF,  Joseph  Vernon, 
William  Kleiner,  and  Matthew  Popham 


The  Defense  Contract  Management  Agency  (DCMA)  and  the  Rolls-Royce 
Corporation  discovered  that  through  close  collaboration  and  teamwork,  day- 
to-day  business  issues  that  previously  required  significant  time  and  resources  to 
be  resolved  instead  quickly  started  to  disappear  or  to  require  less  effort  to  be 
resolved.  This  article  discusses  the  underpinnings  of  a  successful  government  and 
contractor  collaboration  though  the  discussion  of  DCMA's  contract  management 
services,  the  contractor's  environment,  technical  challenges  realized  within 
manufacturing  and  business  systems,  followed  by  a  look  at  problem  solving 
approaches  and  philosophies,  all  of  which  led  to  significant  improvements  in  both 
the  quality  and  on-time  delivery  to  the  warfighters  at  a  fair  and  reasonable  price. 

The  Defense  Contract  Management  Agency  (DCMA)  is  a  combat  support 

organization  within  the  Department  of  Defense  (DoD)  that  provides  worldwide 
contract  management  services  to  all  branches  of  the  U.S.  Armed  Forces  as 
well  as  to  other  government  agencies,  such  as  the  U.S.  Department  of  Energy  and 
National  Aeronautics  and  Space  Administration  (NASA).  In  addition  to  DCMA’s 
global  offices,  the  Agency  consists  of  men  and  women  currently  deployed  overseas 
performing  Contingency  Contract  Administration  Services  (CCAS).  The  DCMA’s 
mission  is  to  “provide  customer-focused  acquisition  life  cycle  and  combat  support 
to  ensure  readiness,  worldwide  24/7”  (DCMA  Vision,  Mission,  and  Goals,  n.d.).  For 
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that  reason,  the  people  at  DCMA  strive  to  make  certain  that  the  warfighter  receives 
the  right  product  at  the  right  time  while  paying  the  right  price.  This  is  accomplished 
through  the  effective  integration  of  various  disciplines,  as  practiced  by  administrative 
contracting  officers  (ACOs),  contract  administrators,  price  analysts,  industrial 
specialists,  quality  assurance  specialists,  engineers,  support  program  integrators, 
earned  value  management,  property  administrators,  and  management  analysts.  A  key 
figure  in  this  challenging  effort  is  the  ACO,  who  functions  as  the  integration  agent 
ensuring  proper  interpretation  and  execution  of  the  contractual  obligations  between 
the  contractor  and  the  government.  In  this  role,  the  ACO  can  effectively  interact 
between  the  contractor  and  the  government  contract  administration  support  team  to 
ensure  the  warfighter’s  requirements  are  fully  met  and  the  delivery  of  products  and 
services  occurs  within  both  cost  and  schedule  at  the  highest  possible  level  of  quality. 
The  DCMA  Aircraft  Propulsion  Operations-Rolls-Royce  is  the  cognizant  DCMA 
office  located  in  the  contractor’s  facility  in  Indianapolis,  Indiana. 


ENVIRONMENT:  CONTRACTOR  MANUFACTURING  FACILITIES 

The  Rolls-Royce  Corporation  in  Indianapolis,  Indiana,  maintains  2.6  million 
square  feet  of  manufacturing  space  and  an  additional  900  thousand  square  feet 
dedicated  to  research  and  development.  The  facility  employs  4,300  workers  and 
delivers  approximately  2  engines  and  $1  million  in  spares  per  day,  and  has  2,700 
active  machine  tools,  7,000  active  part  numbers,  and  $1.5  billion  in  annual  sales. 
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Furthermore,  Rolls-Royce  has  invested  over  $200  million  in  capital  improvements 
since  purchasing  the  facility  in  1995. 

Rolls-Royce  Corporation’s  primary  product,  gas  turbine  engines,  supports  civil, 
military,  energy,  and  marine  applications  worldwide.  Major  customers  include  the 
U.S.  Navy,  Army,  Air  Force,  Marines,  Coast  Guard,  Lockheed  Martin,  Northrop 
Grumman,  Bell,  Boeing,  and  Embraer.  The  facility,  formerly  Allison  Engine 
Company,  has  a  proud  heritage  of  supporting  the  warfighter.  Since  its  inception  in 
1915,  the  company  has  produced  state-of-the-art  engines  that  embody  cutting-edge 
technology.  The  facility  has  produced  over  1 10,000  engines.  From  the  venerable  T56 
turboprop,  used  on  more  than  5000  C-130s  worldwide,  to  the  latest  AE1 107  Liberty 
engine  powering  the  V-22  Osprey  and  the  AE3007  turboprop  utilized  on  the  RQ-4A 
Global  Flawk,  the  tradition  continues.  The  company’s  performance  has  been  validated 
by  the  presentation  of  seven  Collier  Trophies  for  innovation  since  1987. 


TECHNICAL  CHALLENGES 

MANUFACTURING  AND  ON-TIME  DELIVERY 

The  timely  delivery  of  quality  products  is  a  primary  goal  of  the  procurement 
activity,  DCMA,  the  contractor,  and  ultimately,  of  great  significance  to  the  warfighter. 
The  importance  of  this  goal  is  reflected  in  the  DCMA  performance  metrics,  which 
were  adopted  by  Rolls-Royce  senior  management.  In  order  to  achieve  this  goal, 
the  manufacturer  must  employ  adequate  program  management,  master  production 
scheduling,  and  enterprise  resource  planning.  In  today’s  commercial  environment, 
specifications  normally  originate  from  the  Original  Equipment  Manufacturer 
(OEM)  in  response  to  performance  requirements  from  the  government.  To  achieve 
this,  required  performance  tolerances  on  specifications  are  even  tighter  with  little 
or  no  variance.  Additionally,  in  today’s  post-9/1 1  environment  of  multiple  military 
deployments,  increased  demands  of  products  and  services,  raw  material  shortages, 
and  increased  research  and  development  activities,  the  ability  to  deliver  a  flawless 
aircraft  engine  or  component  when  the  government  wants  is  an  ever  increasing 
challenge. 

FINANCIALLY  VIABLE  BUSINESS  SYSTEMS 

The  contractor’s  business  and  financial  health  is  of  major  concern  to  the 
government,  as  a  higher  degree  of  financial  risk  can  mean  increased  costs  to 
the  government.  This  is  especially  true  with  large  contractors,  upon  whom  the 
government  relies  to  provide  major  defense  systems  and  subsystems.  The  assessment 
of  this  financial  risk  to  the  government  is  accomplished  through  the  application  of 
business  systems'  and  Cost  Accounting  Standards  (CAS)2  as  established  by  public 
law.  Adequate  Business  Systems  and  CAS  compliance  means  smoother  and  less 
complicated  proposals  and  pre-awards  through  a  greater  reliance  on  the  contractor’s 
ability  to  track,  estimate,  and  forecast  costs,  which  results  in  lower  risk  to  the 
government  and  earlier  contract  award.  Large  defense  contractors,  such  as  Rolls- 
Royce,  endeavor  to  maintain  “adequate”  business  systems  and  CAS  compliance 
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even  though  adequacy  can  be  somewhat  subjective.  Although  business  systems  are 
highly  regulated,  the  determination  of  an  adequate  business  system  often  rests  on 
the  application  of  the  government’s  interpretation  of  the  regulatory  and  statutory 
language  against  the  contractor’s  current  policies  and  procedures.  Keeping  a 
company’s  business  systems  adequate  is  a  daunting  task  but  the  lack  of  adequate 
business  systems  can  lead  to  the  loss  of  new  government  contracts  and  subcontracts 
as  well  as  possible  withholds  and  decrements  consequently  creating  cash  flow  and 
resource  issues  for  both  the  contractor  and  the  government. 


PROBLEM  SOLVING  APPROACHES 
MANUFACTURING  APPROACH 

Approximately  five  years  ago,  the  leadership  of  Rolls-Royce  and  the  local  DCMA 
realized  that  a  different  approach  was  needed  to  increase  the  timely  delivery  of 
quality  engines  and  components.  Out  of  this  need  was  bom  the  integrated  team 
approach,  which  has  transformed  the  way  the  government  and  the  contractor  do 
business. 

The  first  team  assembled  was  chartered  to  identify  and  eliminate  the  causes  of  late 
delivery.  In  contrast  to  past  practices,  both  DCMA  and  Rolls-Royce  representatives 
were  in  the  room  together  to  discuss  specific  issues  and  solutions.  Each  step  of 
the  process  was  mapped,  resulting  in  a  combined  understanding  of  the  factors 
driving  delivery  excellence  throughout  the  entire  supply  chain  including  delivery 
to  the  end  user.  Tracking  performance  was  accomplished  by  the  Team’s  agreement 
to  use  the  DCMA  On-Time  Delivery  2002  proposed  Performance  Goal  of  70 
percent  as  a  baseline.  Team  members  shared  timely  and  pertinent  information  and 
discussed  solutions  resulting  in  an  atmosphere  of  understanding  and  trust  focused 
on  eliminating  obstacles.  This  joint  collaboration  resulted  in  the  Team  setting  a 
higher  local  on-time  delivery  goal  of  90  percent  for  2006,  of  which  Rolls-Royce 
has  consistently  achieved  a  level  of  on-time  delivery  of  95  percent  or  better.  More 
importantly,  the  process  also  took  the  surprise  factor  out  of  the  equation.  If  a  part  was 
going  to  be  late,  required  rescheduling,  or  had  other  constraints,  both  parties  knew 
about  it  well  in  advance.  This  information  could  then  be  provided  to  the  procurement 
activities  so  that  mitigating  actions  could  take  place  as  required. 

This  approach  garnered  wide  recognition.  First,  the  facility  was  awarded  for 
Outstanding  Supply  Chain  Management  by  the  U.S.  Air  Force  Material  Command  in 
200 1 .  It  was  also  awarded  the  DCMA  Herbert  W.  Homer  Team  Performance  Award- 
Delivery  Management  Integrated  Product  Team  in  2002,  and  the  Rolls-Royce  Chief 
Executive  Quality  Award  in  2003. 

Since  the  first  integrated  team,  many  others  have  been  chartered  with  similar 
results  such  as  quality  assurance,  in  which  Rolls-Royce  and  DCMA  share  audit 
schedules,  findings,  trends,  and  corrective  and  preventative  action  programs.  The 
DCMA  now  accompanies  Rolls-Royce  during  each  of  their  ISO  9001  audits  and 
certifications.  They  also  attend  outbriefs  and  participate  as  corrective  and  preventative 
actions  are  assembled.  Other  teams  perform  Material  Review  Boards  (MRBs),  as 


206 
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well  as  reviews  of  vital  statistics,  scrap,  compliance,  process  initiatives,  and  field 
feedback.  The  DCMA  keeps  Rolls-Royce  informed  about  field  reports  so  there  is  a 
real — time  review  and  both  organizations  stay  proactive  in  dealing  with  any  issue  real 
or  potential  that  may  impact  the  end  user. 

This  integration  has  promoted  the  tenants  of  open  communication,  trust,  and 
continuous  improvement,  all  of  which  are  vital  to  the  success  of  our  combined 
mission  to  provide  the  best  quality  engines  and  commodities  possible. 

BUSINESS  SYSTEMS  SOLUTIONS 

The  purchase  of  the  Indianapolis  facilities  by  Rolls-Royce  in  1995  created  major 
changes  in  the  company’s  business  systems  and  complex  accounting  practices, 
which  continue  to  evolve.  Realizing  that  sound  business  systems  and  accounting 
practices  are  crucial  elements  in  maintaining  and  securing  government  contracts,  the 
government  and  Rolls-Royce  began  in  2004  to  take  a  proactive  approach  to  ensure 
the  company  has  strong  internal  business  and  financial  controls  by  implementing 
a  biweekly  Integrated  Process  Team  (1PT).  Participants  in  this  1PT  include  various 
members  of  Finance  and  Compliance  on  the  part  of  Rolls-Royce  Corporation. 
Government  participants  include  the  DCMA  ACO  and  the  Defense  Contract 
Audit  Agency  (DCAA).  Meetings  include  the  review  of  all  open  issues  and  airing 
concerns  and  observations.  If  system  improvements  are  warranted,  the  process 
begins  immediately  rather  than  awaiting  the  issuance  of  a  DC  AA  draft  audit  report. 
The  1PT  has  experienced  several  instances  where  early  identification  of  issues  and 


207 


DEFENSE  ACQUISITION  REVIEW  JOURNAL 


immediate  evidence  of  contractor  corrective  actions  resolved  the  issue  and  negated 
the  need  for  further  DCAA  involvement.  Moreover,  due  to  the  open  deliberations 
that  occur  within  the  process,  the  ACO  is  in  a  greater  position  to  make  an  informed 
determination  that  is  beneficial  to  both  the  government  and  the  contractor  all  while 
satisfying  regulatory  and  statutory  requirements. 

An  additional  example  of  open  communication  and  transparency  of  company 
processes  include  the  active  participation  in  company  system  conversions.  For 
example,  when  the  company  implemented  a  Plant  Structures  project  to  replace  the 
Billing  and  Material  Management  Accounting  Systems  (MMAS)  on  the  Systems 
Applications  and  Products  in  Data  Processing  (SAP),  DCMA  and  DCAA  were 
invited  to  all  training  sessions  and  steering  committee  meetings.  The  government 
was  also  provided  an  “issues  log”  enabling  them  to  track  the  resolution  of  system 
implementation  issues. 

In  addition  to  IPT  meetings,  the  company’s  Government  Compliance  Manager 
and  the  DCAA  Supervisor  meet  regularly  to  review  all  business  system  status 
reports.  This  meeting  is  directly  related  to  the  IPT  process  and,  again,  allows  for 
early  identification  of  potential  issues,  rather  than  addressing  the  issues  within  an 
audit  report.  The  DCAA’s  timely  reviews,  once  corrective  actions  have  been  taken, 
represent  another  necessary  aspect  both  the  company  and  DCMA  rely  upon. 


COLLABORATION  AND  TEAMWORK 
PROBLEM-SOLVING  PHILOSOPHY 

Up  to  the  moment  when  DCMA  and  Rolls-Royce  decided  to  join  efforts  to  address 
and  solve  issues  in  a  collaborative  fashion,  both  parties  basically  performed  issues 
analysis  and  resolution  independently  of  each  other.  This  approach  was  inefficient 
in  that  a  lot  of  time  and  effort  went  into  solving  issues  in  isolation;  data  and 
information  were  exchanged  in  separate  analytical  environments.  This  method  had 
the  disadvantage  of  requiring  multiple  exchanges  of  data  and  information  without 
the  benefit  of  having  the  same  forum  or  the  right  individuals  with  the  knowledge  and 
expertise  to  provide  answers  and  clarification  in  a  real-time  basis.  Therefore,  solving 
issues  was  a  lengthy  and  time-consuming  endeavor  not  sufficiently  agile  to  resolve 
issues. 

Once  both  parties  decided  to  join  efforts  and  work  collaboratively  in  the  early 
identification  and  resolution  of  issues,  efficiencies  were  realized  immediately.  Both 
teams  as  well  as  the  warfighting  customers  started  to  see  the  tangible  benefits  of  this 
approach  though  increased  on-time  delivery,  a  reduction  of  quality  issues  in  the  field, 
and  a  better  understanding  of  the  contractor’s  business  and  financial  health.  What 
actually  existed  after  the  new  collaboration  efforts  began  was  a  collectively  shared 
sense  of  purpose  augmented  by  the  empowerment  given  to  both  DCMA  and  Rolls- 
Royce  teams  by  their  senior  management,  thus  enabling  the  teams  to  tackle  the  issues 
and  allocate  resources  to  solve  them. 
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Gigantic  strides  have  been  made  in  the  overall  relationship  between  DCMA  and 
Rolls-Royce,  primarily  due  to  the  realization  that  (a)  both  are  working  toward  a 
common  goal  of  providing  the  best  product  to  the  end  user  at  the  right  time  and  for 
the  right  price,  and  (b)  that  in  today’s  environment  of  declining  resources,  it  is  better 
to  offensively  pool  assets  towards  common  goals  rather  than  to  use  assets  defensively 
against  each  other.  Such  a  relationship  must  be  built  on  trust  and  an  understanding 
that  both  the  government  and  the  contractor  share  an  immense  obligation  and 
responsibility  to  provide  the  best  quality  product  to  the  end  user  when  and  where 
it  is  needed.  However,  trust  and  understanding  play  an  even  greater  role  where  the 
goals  of  the  government  and  the  contractor  differ,  such  as  disagreements  about  price, 
delivery  schedules,  or  policies,  such  as  the  Berry  Amendment  Specialty  Metals 
Clause,3  Critical  Safety  Items  (CSI),4  and  Business  System  Status.  In  such  situations, 
the  need  for  open  and  honest  communication  is  imperative  to  keep  product  moving, 
the  contractor  paid,  and  the  opportunity  granted  for  new  business  and  corporate 
growth.  Hidden  agendas  and  unilateral  posturing  lead  only  to  an  environment  of 
suspicion  and  lack  of  cooperation  whereby  everyone,  especially  the  end  user,  loses 
through  the  re-alignment  of  resources  in  a  defensive  posture. 

Most  importantly,  both  government  and  corporate  senior  management  have  given 
their  people  the  authority  to  exercise  prudent  business  judgment  in  the  resolution  of 
issues  as  well  as  binding  the  parties  in  bilateral  agreements.  Conversely,  along  with 
such  authority  comes  accountability  as  well  as  the  realization  of  the  consequences 
of  honoring  agreements  and  commitments.  Thus,  the  framework  of  government 


FIGURE  3.  ROLLS-ROYCE  2100D2  TURBO  PROP  ENGINE 
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and  corporate  collaboration  is  very  similar  for  both  the  manufacturing  and  business 
systems  environments.  The  lines  of  communications  have  been  established  through 
senior  management  sanctioned  IPTs,  giving  the  individual  teams  the  authority  and 
accountability  to  work  out  complex  issues  and  implement  prudent  business  corrective 
actions.  This  in  turn  leads  to  the  ultimate  success  and  satisfaction  of  all  interested 
parties,  while  ensuring  the  mission  is  accomplished. 

There  are  several  areas  in  which  both  the  government  and  the  contractor  must 
ensure  that  the  right  conditions  exist  if  any  collaborative  effort  is  to  have  a  fertile 
ground  to  grow.  These  areas  include: 

1.  People:  It  is  management’s  responsibility  to  ensure  that  the  right  human  capital  is 
employed.  Both  entities  must  chose  to  allocate  individuals  who  have  the  proper 
background  and  right  level  of  experience  to  function  effectively  in  this  teaming 
effort. 

2.  Framework:  Both  parties  must  agree  to  a  common  structure  in  which  the 
collaborative  relationship  will  exist.  This  framework  spells  out  the  type  of  teams 
needed,  who  is  needed  on  the  team,  the  scope  and  authority  of  the  team,  and  the 
team’s  oversight.  In  this  case,  the  Manufacturing  Team  is  made  up  of  both  Rolls- 
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Royce  and  DCMA  Quality  Assurance  representatives  while  the  Business  and 
Financial  Team  is  represented  by  contract  and  accounting  members  from  Rolls- 
Royce,  DCMA,  and  DCAA. 

3.  Rules  and  Protocols:  These  provide  for  a  disciplined  and  structured  approach 
to  solving  issues.  Before  both  parties  decide  to  join  efforts  as  one  integrated 
team,  each  team  has  its  own  set  of  rules  and  protocols  to  identify,  work,  and 
solve  problems.  This  approach  has  often  resulted  in  a  cognitive  disagreement 
between  the  teams,  thus  generating  a  psychological  state  of  conflict.  Now,  both 
the  manufacturing  and  business  teams  are  aware  and  follow  required  rules  and 
protocols,  so  that  each  team  may  focus  on  the  specific  issue  rather  than  the 
process. 

4.  Trust:  One  of  the  pillars  of  success  to  the  DCMA  and  Rolls-Royce  collaboration 
experience  is  trust.  Past  interactions  between  the  government  and  Rolls-Royce 
were  plagued  by  both  parties  arriving  at  conclusions  within  a  separate  analytical 
environment.  This  situation  proved  fertile  ground  for  distrust  between  the  parties, 
because  neither  one  had  an  opportunity  to  openly  discuss  specific  details  or 
particular  circumstances  associated  with  the  problem.  By  having  both  teams 

in  the  same  room  at  the  same  time  with  the  right  individuals,  confidence  and 
reliance  on  each  member  was  established.  This  was  reinforced  through  the  open 
and  honest  sharing  of  information  and  the  commitment  of  team  members  to  work 
through  complex  issues  with  a  common  goal  resulting  in  a  heightened  degree  of 
trust  and  understanding. 

5.  Senior  Leadership  Support:  Once  a  decision  was  made  to  establish  the  IPT,  both 
teams’  senior  management  remained  engaged  and  closely  involved  in  the  IPT 
implementation  and  execution.  Human  resources  were  exclusively  allocated  and 
dedicated  to  the  IPT,  and  team  members  are  fully  empowered  to  make  decisions 
and  commitments  to  see  that  issues  are  completely  solved.  This  support, 
combined  with  the  increased  level  of  trust,  is  pivotal  to  the  successful  DCMA/ 
Rolls-Royce  teaming  effort. 


CONCLUSIONS 

There  are  several  characteristics  that  can  be  identified  as  key  to  the  success 
that  DCMA  and  Rolls-Royce  have  experienced  through  their  journey  to  better 
collaboration  and  teamwork  in  support  of  the  warfighter.  Whereas  in  the  past  both 
teams  were  individually  less  efficient  at  solving  issues,  the  new  collaborative 
approach  yields  quantifiable  benefits  for  everyone  (e.g.,  timely  delivery  and  financial 
stability).  First  and  foremost,  there  must  be  full  commitment  and  sponsorship  from 
each  organization’s  senior  leadership.  It  is  absolutely  critical  that  each  organization 
identify  and  assign  the  right  human  talent  and  resources.  Second,  there  must  be 
a  clear  set  of  rules  and  protocols  for  the  team  to  follow.  Up-front  definition  of 
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procedures,  roles,  and  responsibilities  ensures  the  efficiency  of  every  team  member 
through  concentration  on  the  issue  rather  than  the  process.  This  helps  the  team 
stay  clear  of  misunderstandings  and  potential  disappointments  that  can  corrode  the 
synergy,  cohesiveness,  and  trust  that  takes  a  lot  of  time  and  effort  to  develop.  Finally, 
it  is  absolutely  necessary  that  the  team  have  the  time,  resources,  and  authority  needed 
to  execute  developed  plans  and  strategies. 


In  tontlusion,  the  DCMA  and  Rolls -Roy re  experiente 
demonstrates  that,  when  the  government  and  tontrattor 
tome  together  in  (ollaboration  for  the  positive  resolution  of 
issues ,  the  end  result  is  a  win-win  situation. 


In  conclusion,  the  DCMA  and  Rolls-Royce  experience  demonstrates  that,  when  the 
government  and  contractor  come  together  in  collaboration  for  the  positive  resolution 
of  issues,  the  end  result  is  a  win-win  situation.  The  warfighters  are  provided  with 
what  they  need,  when  they  need  it.  The  government  can  ensure  proper  expenditure 
of  taxpayer  dollars  as  well  as  a  secure  vendor  base,  and  the  contractor  can  enjoy 
the  opportunity  for  corporate  growth.  This  example  of  successful  collaboration  and 
teamwork  can  be  used  as  the  basis  to  develop  similar  working  relationships  within 
other  components  of  DoD,  particularly  in  the  acquisition  community. 
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ENDNOTES 


1.  There  are  currently  ten  primary  business  systems.  These  include  accounting, 
billing,  budget  and  planning,  compensation,  estimating,  indirect  and  other 
direct  cost  (ODC)  internal  controls,  general  Information  Technology  (IT) 
controls,  labor,  purchasing,  and  material  management,  which  are  audited  by  the 
government  on  a  recurring  cycle  in  accordance  with  the  Defense  Contract  Audit 
Agency  (DCAA)  Internal  Controls  Audit  Planning  System  (ICAPS). 

2.  Rolls-Royce  is  considered  a  “Full  Coverage”  contractor  as  described  in  FAR 
9903.201-2  and  is  therefore  required  by  Public  Law  100-679  (41  U.S.C.  422)  to 
comply  with  all  Cost  Accounting  Standards  specified  in  Part  9904. 

3.  The  Berry  Amendment,  as  stated  in  10  U.S.C.  2533a,  originated  in  1941  and 
was  established  to  address  concerns  over  the  procurement  of  domestic  materials 
for  use  in  U.S.  military  items.  The  Specialty  Metals  Clause,  as  stated  in  DEARS 
252.225-7014,  was  added  to  the  Berry  Amendment  in  1972  and  addresses  the 
use  of  such  specialty  metals  as  titanium,  zirconium,  and  steel  alloys  in  U.S. 
military  commodities.  Recent  events  between  metal  producers  and  Original 
Equipment  Manufactures  (OEMs),  especially  the  aeronautical  industry,  have 
resurfaced  this  statutory  requirement,  as  well  as  interim  governmental  guidance 
pursuant  to  acceptance,  exceptions,  and  monetary  withholds  based  on  the  value 
of  nonconforming  parts. 

4.  Critical  Safety  Items  (CSIs),  as  described  in  DEARS  209.270,  are  parts, 
assemblies,  or  equipment  for  aircraft  or  aviation  systems  of  which  catastrophic 
or  critical  failure  could  result  in  (a)  loss  of  or  serious  damage  to  aircraft  or 
weapon  system,  (b)  unacceptable  risk  of  personal  injury  or  loss  of  life,  or  (c) 
uncommanded  engine  shutdown  that  jeopardizes  safety.  The  Joint  Aeronautical 
Logistics  Commanders  (JALC)  have  implemented  specific  directives  pursuant 
to  the  National  Defense  Authorization  Act  (Public  Law  1 08- 1 36),  of  which 
Section  802  requires  the  Secretary  of  Defense  to  prescribe  a  policy  for  the 
quality  assurance  of  aviation  CSIs.  Therefore,  DCMA  is  tasked  with  the  central 
role  of  evaluating  the  CSI  characteristics  or  features  that  have  been  identified 
by  the  respective  Service  Engineering  Support  Activity  (ESA).  For  any  part  that 
has  been  identified  by  ESA  as  a  CSI  with  critical  characteristics,  the  Product 
Assurance  Specialist  will  perform  surveillance  activities  using  a  Quality 
Release  Level  that  has  been  specified  within  the  agency’s  Product  Assurance 
Instructions  and  Guidance  documents.  If  a  part  has  been  identified  as  a  CSI  but 
critical  characteristics  or  features  have  not  been  identified  by  the  ESA,  DCMA 
is  responsible  for  evaluating  those  processes  that  are  directly  related  in  the 
manufacture  of  the  CSI  and  have  been  deemed  important  by  the  ESA. 
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A  BUSINESS  MODEL 
FOR  DEFENSE 
ACQUISITION  UNDER 
THE  MODULAR  OPEN 
SYSTEMS  APPROACH 


Eugene  Gholz 


This  article  briefly  describes  a  business  model  that  companies  may  apply  to 
develop,  produce,  and  sell  avionics  to  the  Department  of  Defense  under  a 
Modular  Open  Systems  Approach  (MOSA).  Recent  acquisition  reforms  have 
encouraged  the  use  of  MOSA,  and  the  defense  industry  will  need  to  adapt 
to  the  new  style  of  acquisition.  A  business  model  summarizes  the  way  a  firm 
earns  profits  sufficient  to  remain  in  business:  it  describes  core  competencies, 
principal  activities,  cost  structure,  and  expected  revenue  stream.  The  model 
proposed  here  suggests  that  firms  can  succeed  under  the  new  framework, 
but  that  MOSA  entails  some  drawbacks  for  both  industry  and  the  government 
that  may  limit  its  applicability  to  a  relatively  small  subset  of  programs. 


Over  the  past  several  years,  the  Department  of  Defense  (DoD),  the  military 
services,  and  the  defense  industry  have  sponsored  a  good  deal  of  research 
about  the  technical  aspects  of  a  Modular  Open  Systems  Approach  (MOSA) 
to  the  acquisition  of  avionics.1  The  results  have  shown  that  while  some  technical 
hurdles  still  remain,  business  issues  may  be  bigger  barriers  than  technical  ones  to  the 
implementation  of  MOSA.2  Advocates  of  MOSA  must  show  that  the  new  approach 
will  solve  some  problems  for  the  defense  industry  as  well  as  for  its  military  customers 
and  that  the  companies  can  be  at  least  as  profitable  under  MOSA  as  under  traditional 
acquisition  strategies.  Firms  in  the  defense  industry  are  relatively  comfortable  with 
their  traditional  business  model,  and  they  require  business  analysis  to  give  them  an 
incentive  to  aggressively  pursue  change.  They  are  generally  quite  ready  to  cooperate 
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with  their  customers’  initiatives — in  fact,  responsiveness  to  the  unique  military 
customer  is  a  hallmark  of  successful  defense  companies  (Gholz  &  Sapolsky,  1999- 
2000) — but  the  customers  need  to  put  their  request  for  such  cooperation  in  the 
business  language  of  company  decision  makers. 


A  business  model  summarizes  the  way  a  firm  earns 
profits  suffident  to  remain  in  business:  it  describes  tore 
tompetenties,  printipal  activities,  tost  strutture,  and 
expetted  revenue  stream. 


This  article  briefly  describes  a  business  model  that  companies  may  apply  to 
develop,  produce,  and  sell  avionics  to  the  DoD  under  MOSA.  A  business  model 
summarizes  the  way  a  firm  earns  profits  sufficient  to  remain  in  business:  it  describes 
core  competencies,  principal  activities,  cost  structure,  and  expected  revenue  stream. 
Defense  firms  organized  for  “business  as  usual”  will  find  a  transition  to  MOSA 
difficult,  and  they  may  prefer  to  create  new  MOSA-oriented  divisions  rather  then 
attempting  to  transform  the  culture  of  established  organizations  (Christensen  & 
Raynor,  2003).  Even  if  the  military  adopts  MOSA  for  many  acquisition  projects, 
the  traditional  approach  will  remain  in  force  for  many  high-value  projects  to  which 
MOSA  is  not  well  suited;  defense  firms  should  maintain  their  existing  structures  to 
pursue  non-MOSA  acquisitions.  But  defense  firms  also  stand  a  reasonable  chance  to 
do  profitable  business  on  some  avionics  projects  under  a  MOSA  business  model.  In  a 
capitalist  economy,  a  reasonable  chance  of  profitable  business,  and  not  a  guarantee,  is 
exactly  what  investors  and  CEOs  hope  to  find. 


THE  TRADITIONAL  DEFENSE  INDUSTRY  MODEL 

During  the  post- World  War  II  era,  the  American  defense  industry  developed 
a  specialized  business  model,  especially  for  prime  contractors  (Gansler,  1995; 
McNaugher,  1989).  Generally  speaking,  today’s  defense  companies  closely  follow 
directives  from  their  military  customers,  developing  customized  products  with 
attributes  specified  in  advance  by  the  buyer.  The  buyer  pays  the  development  costs  up 
front  or  in  stages  during  the  development  process.  Because  defense  firms  primarily 
invest  the  customer’s  money  rather  than  their  own  in  research  and  development, 
the  firms  have  a  limited  role  in  choosing  the  technological  trajectory  that  they  will 
pursue  (Dombrowski  &  Gholz,  2006).  Consequently,  they  have  relatively  less  skill 
in  technology  management  than  companies  in  other  industries  that  emphasize 
innovation  to  a  comparable  extent.  The  most  responsive  defense  companies  tend  to 
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be  the  most  successful:  executives  and  managers  constantly  reassure  their  customers 
that  the  customers’  interests  are  the  firm’s  top  priority  and  adapt  the  firm’s  business 
processes  to  remain  relatively  efficient  while  following  complex  and  intrusive 
acquisition  regulations. 

In  the  traditional  defense  business  model,  firms  (especially  prime  contractors) 
are  rewarded  with  a  relatively  stable  income  stream.  Their  close  relationship  to  the 
military  customer  gives  them  a  relatively  low  level  of  risk,  comparable  to  the  steady, 
low-risk  business  of  a  regulated  public  utility  (Gholz  &  Sapolsky,  1999-2000). 


MOSA  AND  ITS  GOALS 

As  the  American  military  increasingly  relies  on  information  technology  in  its  new 
weapon  and  support  systems,  the  traditional  business  model  seems  less  appropriate 
than  it  used  to.  Advocates  have  proposed  MOSA  for  avionics  acquisition  to  adapt  to 
the  military’s  plans  for  network-centric  warfare,  to  take  advantage  of  technological 
opportunities  that  have  developed  in  the  commercial  information  technology 
business,  and  to  improve  the  sustainability  of  military  equipment  as  the  product  cycle 
for  avionics  has  raced  ahead  of  the  long  life  cycle  of  the  military’s  platforms.  They 
also  hope  that  MOSA  can  exploit  the  advantages  of  competition  to  control  the  soaring 
costs  of  systems  acquisition. 


Under  its  furrent  business  model,  the  defense  industry 
tends  to  tustomize  products  on  a  platform-by-platform 
basis,  but  with  transformation,  each  firm's  proprietary 
technical  solutions  need  to  interface  with  other  firms' 
proprietary  products. 


The  shift  to  network-centric  warfare  is  one  of  the  major  drivers  for  MOSA. 

The  American  military  expects  to  use  the  next  generation  of  equipment  to  share 
information  across  the  battlespace  more  than  ever  before.  Under  its  current  business 
model,  the  defense  industry  tends  to  customize  products  on  a  platform-by-platform 
basis,  but  with  transformation,  each  firm’s  proprietary  technical  solutions  need 
to  interface  with  other  firms’  proprietary  products.  Even  if  firms  do  not  share 
the  technologies  that  underlie  their  products’  internal  performance  and  only  the 
interface  designs  are  widely  disseminated,  defense  systems  as  a  whole  will  become 
less  proprietary.  This  trend  emphasizes  the  “open  systems”  part  of  MOSA:  open 
systems  are  “integrated  from  elements  provided  by  multiple  sources”  based  on 
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“nonproprietary  interface  standards”  (Committee  on  Aging  Avionics  in  Military 
Aircraft,  2001,  pp.  32-33).  At  the  technical  level,  MOSA  requires  decisions  about 
what  those  interface  standards  should  be.  Meanwhile,  MOSA  calls  for  defense 
industry  design  teams  to  focus  on  learning  open  interface  standards  and  to  contribute 
their  expertise  to  the  process  of  choosing  and  updating  the  standards. 

The  increasing  military  interest  in  information  technology  (IT)  has  also  naturally 
drawn  attention  to  the  commercial  IT  industry.  Since  the  1990s,  many  people 
have  observed  that  commercial  IT  tends  to  be  more  advanced  than  military  IT. 
Commercial  businesses  also  offer  nearly  continuous  innovation  to  their  customers 
(Alic,  Branscomb,  &  Brooks,  1992),  enabled  by  modularity  of  commercial  products. 
Commercial  customers  can  replace  parts  of  their  systems  when  new  component 
technologies  become  available  rather  than  paying  to  replace  the  entire  system. 
Internal  changes  can  increase  a  module’s  capabilities  or  simply  reduce  the  cost  of 
production  or  operation  of  a  module  at  the  same  level  of  performance  (Committee  on 
Aging  Avionics  in  Military  Aircraft,  2001). 


The  increasing  military  interest  in  information 
technology  (IT)  has  also  naturally  drawn  attention  to 
the  commercial  IT  industry. 


In  the  new  language  of  defense  acquisition,  modular  design  facilitates  spiral 
development.  Because  modules  can  be  replaced  one  by  one  and  can  be  taken  from 
existing  systems  and  combined  in  new  ways  to  produce  new  systems,  the  customer 
need  not  define  all  of  the  performance  requirements  for  a  system  in  advance.  Instead, 
the  customer  can  experiment  with  an  initial  version  to  reveal  which  modules  most 
tightly  constrain  overall  system  performance.  That  experience  will  then  allow  the 
customer  to  define  reasonable  performance  requirements  for  the  next  iteration  of 
equipment  and  to  focus  resources  in  the  next  development  spiral  on  improving  key 
modules  without  redesigning  the  entire  system. 

In  the  same  way,  modular  design  also  facilitates  technology  refresh,  reducing 
the  need  to  maintain  obsolete  parts  in  legacy  systems.  If  a  system  is  designed  to  be 
modular,  obsolete  parts  can  be  thrown  away  and  replaced  with  newer,  cheaper,  more 
capable  parts.  As  long  as  the  new  module  has  the  same  external  compatibility,  it 
will  not  matter  whether  the  internal  components  are  the  same  as  the  old  ones.  The 
military  will  no  longer  need  to  stockpile  replacement  parts  or  maintain  production 
lines  that  freeze  technology  at  a  particular  point  in  time.  Recognizing  the  fast  rate 
of  improvement  in  computer  hardware  and  software  and  other  electronics,  many 
modular  components  can  be  designed  with  the  assumption  that  they  likely  will  be 
replaced  through  a  technology  insertion  program.  This  design  change  will  have  the 
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added  benefit  of  eliminating  the  need  for  expensive  efforts  to  guarantee  performance 
through  years  of  use,  because  many  parts  will  be  “disposable”  after  relatively  short 
life  spans. 

The  combination  of  modularity  with  open  systems  may  gain  additional  benefits  of 
competition,  spurring  innovation,  and  controlling  costs.  Under  the  traditional  defense 
acquisition  approach,  the  customer  is  often  locked  in  to  buying  parts  from  a  sole 
source  (the  original  producer).  Some  of  the  high  cost  of  maintenance — probably  a 
substantial  fraction — is  driven  by  the  real  overhead  cost  of  maintaining  inventories 
and  keeping  old  production  lines  open.  But  many  politicians,  military  leaders,  and 
analysts  fear  that  the  cost  is  driven  up  a  significant  additional  margin  by  the  reliance 
on  a  monopoly  supplier  (Kovacic,  1999).  Even  though  government  auditors  try  hard 
to  monitor  actual  costs,  have  tremendous  access  to  the  firm’s  cost  data,  and  strive 
to  avoid  paying  “too  much,”  the  lack  of  competition  once  a  supplier  is  guaranteed 
a  long-term  market,  especially  for  sales  of  replacement  parts,  makes  the  buyer 
vulnerable.  Whether  or  not  defense  firms  truly  exploit  their  customers,  American 
political  and  economic  leaders  distrust  monopolies.  The  potential  to  eliminate 
monopolistic  spare  parts  sales  is  a  significant  symbolic  benefit  of  MOSA. 


INTUITIVE  SUMMARY  OF  THE  BUSINESS  MODEL 

The  MOSA  redraws  the  division  of  labor  between  the  government  and  industry 
in  military  systems  development:  industry  will  lead  the  relationship  to  a  greater 
extent  than  in  the  traditional  model.  Instead  of  announcing  detailed  performance 
requirements  for  major  systems  acquisitions  or  upgrades,  under  MOSA  the  customer 
will  only  publish  “roadmaps,”  general  descriptions  that  link  the  performance 
characteristics  of  new  systems  to  the  military’s  evolving  strategy  and  doctrine.  Firms 
will  create  the  detailed  specifications  and  develop  systems  and  modules  that  might 
interest  the  customer. 

Companies  will  be  able  to  propose  improvements  to  modules  or  systems 
whenever  they  have  a  new  product  ready,  developed  on  the  companies’  preferred 
schedule — influenced,  of  course,  by  contacts  with  the  military  buyers.  This  company- 
led  proposal  process  will  not  work  for  platforms  or  major  avionics  systems  that 
Congress  must  fund  on  a  line  item  basis,  but  it  will  be  easier  to  implement  for  smaller 
systems  and  system  components.  The  MOSA  will  be  easiest  in  the  maintenance  and 
upgrade  budgets,  which  rarely  attract  the  attention  of  political  leaders  and  already  are 
supported  through  a  process  that  delegates  more  discretion  to  program  managers. 

To  maximize  the  advantage  of  MOSA  for  technology  insertion  and  spiral 
development,  the  military  will  buy  small  batches  of  parts  or  systems.  From  time  to 
time — perhaps  on  a  regular  schedule,  perhaps  when  installed  equipment  breaks,  or 
perhaps  in  response  to  unsolicited  offers  from  defense  firms — companies  will  be 
able  to  bid  on  a  batch  of  sales.  For  each  bid,  the  companies  will  have  to  provide  a 
complete  description  of  the  performance  characteristics  of  their  products,  a  unit  price, 
and  a  number  of  units  for  which  that  price  will  be  valid.  Firms  will  have  the  option 
to  offer  the  same  equipment  that  they  offered  in  the  previous  iteration  of  competition 
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(at  the  same  price  or  a  new  one),  or  they  may  offer  a  new  product  that  incorporates 
additional  technological  progress.  The  customer  might  purchase  the  new  batch  from 
the  incumbent  supplier,  or  the  customer  may  buy  the  new  batch  from  one  of  the 
incumbent’s  competitors.  The  only  advantage  that  the  incumbent  will  enjoy  in  the 
competition  would  come  from  whatever  benefit  it  had  gained  from  learning-by-doing 
or  economies  of  scale  on  production  of  the  previous  batch. 

The  government  may  ask  for  minor  modifications  of  the  product  as  part  of  the 
contracting  process,  but  the  more  that  those  modifications  are  requested,  the  less  the 
customer  will  be  able  to  benefit  from  MOSA.  Ultimately,  if  the  customer  requires  too 
many  modifications,  it  should  apply  the  traditional  acquisition  model  rather  than  the 
MOSA  for  the  project  in  question. 


PRIME  CONTRACTOR  CORE  COMPETENCIES 

The  MOSA  will  require  leading  avionics  companies  to  have  two  core 
competencies:  technology  management  and  portfolio  management.  Firms  will 
still  need  to  nurture  skills  in  product  design  and  manufacturing,  just  as  they  have 
under  the  traditional  defense  business  model,  but  their  current  core  competency  in 
responding  to  intrusive  government  oversight  and  regulation  will  fade  in  importance. 

Firms  will  use  their  core  competency  in  technology  management  to  decide  how 
much  to  invest  in  research  and  development  (R&D)  and  how  much  to  charge  per 
unit,  given  their  products’  performance  specifications.  This  skill  set  is  not  entirely 
unfamiliar,  as  defense  firms  already  project  cost  and  performance  when  they  make 
paper  proposals  early  in  competitive  projects.  But  under  MOSA,  the  firms  will  have 
substantially  more  discretion  and  will  face  significantly  more  complex  technology 
management  problems. 

Most  important,  under  MOSA  the  defense  industry  will  choose  what  product 
improvements  to  offer  and  when  those  new  products  should  be  developed.  Under  the 
current  system,  warfighters  sometimes  suggest  unrealistic  hopes  for  new  technology, 
because  their  expertise  tends  to  emphasize  the  military  arts  rather  than  science  and 
technology.  The  balance  of  emphasis  in  acquisition  planning  has  especially  shifted 
in  favor  of  military  rather  than  technical  factors  in  recent  years,  as  Combatant 
Commanders  have  gained  a  more  prominent  role  in  the  process.  This  customer 
influence  often  drives  the  pace  and  direction  of  investment  in  ways  that  will  not  yield 
the  greatest  marginal  research  benefit,  increasing  R&D  costs.  Buyers  also  change 
their  minds  about  what  they  want,  leading  to  poorly  planned  investment  programs  or 
to  the  “hurry  up  and  wait”  syndrome  that  can  sometimes  plague  systems  development 
(McNaugher,  1989).  Under  MOSA,  by  contrast,  the  firms  will  decide  on  the 
trajectory  of  technological  change  that  they  should  invest  in,  giving  more  influence  to 
the  “natural”  direction  in  which  technological  improvements  are  available. 

Firms  should  be  more  sensitive  to  the  financial  costs  of  lurches  in  their  research 
program.  Their  decisions  will  not  be  driven  purely  by  scientific  estimates  of  which 
technical  problems  are  most  readily  solved,  because  company  executives  are  driven 
by  the  profit  motive  and  because  they  will  still  have  to  consider  what  technological 
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improvements  they  think  that  their  customers  will  want  to  buy.  Prime  contractors 
will  also  still  consider  their  customer’s  social  goals  in  their  analyses  of  alternatives.3 
But  ideally,  defense  firms,  which  often  employ  former  warfighters  in  their  business 
development  departments,  will  be  in  a  better  position  than  the  military  customer  to 
understand  simultaneously  both  the  technical  and  warfighting  influences  on  weapons 
development.  As  a  result,  the  MOSA  business  model  should  increase  the  technical 
payoff  and  reduce  the  cost  of  R&D  investment. 

For  the  defense  industry,  the  increased  control  of  R&D  investment  comes  at  a 
price.  First,  the  military  customer  will  not  always  be  interested  in  the  technological 
improvements  that  the  defense  industry  develops  and  offers.  Firms  will  do  their 
best  to  understand  the  military’s  needs,  and  under  MOSA  the  military  will  work 
with  industry  to  develop  roadmaps  that  identify  desirable  product  improvements. 

But  forward-looking  roadmaps  will  always  imperfectly  predict  warfighters’  needs 
and  political  leaders’  budget  priorities,  so  the  customer  may  decline  any  particular 
product-price  offer  that  a  firm  makes. 

Second,  because  the  companies  gain  control  of  investment  decisions  under  MOSA, 
they  will  have  to  put  their  own  money  on  the  line.  At  least  in  the  idealized  MOSA 
business  model,  the  companies  will  offer  off-the-shelf  products  to  the  buyer — that 
is,  products  that  they  have  developed  on  their  own  prior  to  offering  them  for  sale.4 
Development  cost  will  be  figured  into  the  price  at  which  the  new  product  is  offered 
for  sale,  as  it  is  in  commercial  markets.  The  buyer  will  bear  little  technological  risk, 
because  the  basic  performance  characteristics  of  the  already  developed  device  will 
be  well  understood  at  the  time  of  the  sale;  the  remaining  uncertainty  will  focus  on 
how  the  warfighter  will  actually  use  the  device.  In  essence,  the  MOSA  model  gives 
companies  control  over  their  investment  decisions,  allowing  them  to  advance  their 
technological  core  competencies,  but  it  increases  the  technological  and  market  risk 
that  they  bear. 


To  fate  the  intrease  in  risk  that  tomes  with  the  MOSA 
business  model \  firms  will  need  to  intrease  their  finandal 
tompetenty  to  manage  a  portfolio  of  tethnology. 


To  face  the  increase  in  risk  that  comes  with  the  MOSA  business  model,  firms  will 
need  to  increase  their  financial  competency  to  manage  a  portfolio  of  technology. 
Firms  should  expect  to  lose  most  of  the  frequent  competitions  that  they  enter.  These 
losses,  though,  will  not  hurt  too  much  because  each  competition  only  offers  the 
opportunity  to  sell  a  small  batch  of  products  to  the  military.  Finance  specialists  in 
the  defense  industry  will  try  to  amortize  the  cost  of  R&D  investment  in  all  of  the 
firm’s  new  technologies  into  the  prices  bid  on  the  competitions  that  the  firm  ends  up 
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winning.  The  winning  bids  must  pay  for  the  successful  product  developments,  the 
dead-end  research  projects,  and  the  products  that  for  whatever  reason  the  customer 
has  chosen  not  to  buy.  Because  the  customer  will  only  buy  a  small  batch  at  a  time,  the 
suppliers  will  not  be  able  to  earn  a  return  on  all  of  their  investment  on  any  individual 
sale.  Instead,  they  will  have  to  incorporate  into  their  pricing  strategies  the  probability 
that  they  will  also  win  the  follow-on  contract.  The  pricing  strategy  and  portfolio 
management  decisions  that  MOSA  asks  of  the  defense  industry  are  quite  complex. 

Through  a  combination  of  financial  instruments,  good  market  research,  and  sound 
competitive  intelligence,  defense  firms  under  the  MOSA  business  model  should  be 
able  to  bear  the  risk  and  earn  a  profit.  When  the  current  defense  industry  business 
model  evolved  during  the  Cold  War,  these  techniques  were  less  sophisticated  than 
they  are  today.  Today’s  firms  stand  a  much  better  chance  of  succeeding  at  the  complex 
business  strategy  decisions  than  they  would  have  decades  ago. 


PRIME  CONTRACTOR  TASKS 

Prime  contractors’  primary  business  activities  will  include  routine  collaboration 
with  the  customer  on  roadmapping  the  trajectory  of  technology,  new  product 
development  that  incorporates  as  many  standardized  modules  as  possible,  and 
bidding  on  a  plethora  of  small  contracts  to  spread  technology  and  market  risk  over  a 
broad  portfolio. 

Technology  roadmaps  are  essential  to  MOSA,  because  firms  need  a  simple  way 
to  understand  what  their  customers  want  from  innovation.  If  firms  are  to  choose 
how  to  invest  their  own  R&D  money,  they  will  need  a  reasonable  expectation  that 
their  customers  will  buy  the  products  that  result  from  any  laboratory  successes. 
Roadmaps  provide  broad  guidelines  and  set  headline  goals  but  do  not  set  particular 
investment  priorities  or  product  specifications.  They  draw  on  the  military’s 
operational  experience,  simulations  that  try  to  model  the  future  strategic  and  tactical 
environment,  and  technical  advice  from  military  laboratories  and  defense  contractors. 

In  the  past,  firms’  principal  contribution  to  their  customers’  technology  planning 
was  informal.  Firms  hired  retired  military  officers  who  could  interface  easily  with 
their  active  duty  counterparts.  Today,  the  firms’  role  in  roadmapping  is  already 
expanding  and  becoming  more  formal.  Some  contractors  have  built  sophisticated 
computer  simulation  systems  that  they  use  for  strategic  planning  (and  marketing). 
Under  MOSA,  a  sale  by  a  prime  contractor  will  include  a  computer  model  of 
the  product’s  behavioral  characteristics,  a  model  that  can  be  plugged  into  future 
simulations  and  thereby  contribute  to  future  roadmaps.  Some  program  offices  (e.g., 
the  Army’s  Future  Combat  Systems)  have  already  used  the  contractors’  simulation 
systems  to  better  evaluate  alternative  project  definitions  and  investment  plans.  This 
collaborative  process  will  become  routine  under  MOSA.  Neither  the  customers  nor 
the  suppliers  will  be  able  to  create  sensible  roadmaps  on  their  own;  collaboration  on 
modeling  and  simulation  will  be  a  key  task. 

Prime  contractors  already  advertise  their  main  activity  as  systems  integration 
rather  than  manufacturing,  and  MOSA  will  reinforce  the  importance  of  systems 


224 


integration  (Gholz,  2003).  In  fact,  the  intellectual  emphasis  of  the  modular  open 
system  approach  suggests  that  prime  contractors  should  develop  their  products  by 
incorporating  as  many  already-developed  component  modules  as  they  can,  given 
technical  constraints.  Using  such  off-the-shelf  contributions  from  Tier  2  suppliers 
will  help  reduce  each  prime’s  up-front  investment  in  in-house  R&D  and  tooling,  and 
the  incorporation  of  such  off-the-shelf  modules  will  also  reduce  total  system  cost  by 
allowing  the  subcontractors  to  plan  to  spread  development  costs  for  their  modules 
across  several  final  product  lines.  Through  this  process,  the  primes’  key  proprietary 
knowledge  will  increasingly  consist  of  their  design  team  philosophies  and  their  trade 
secret  ways  of  drawing  together  subcontractors’  modules  into  optimally  designed 
systems  (Drezner,  et  al.,  1992). 

Prime  contractors  will  also  be  responsible  for  partitioning  the  functionality  of 
systems  into  modules.  If  a  system  is  a  set  of  “black  box”  modules  that  work  together, 
someone  has  to  decide  what  functions  belong  in  each  black  box,  what  processing 
has  to  be  accomplished  internally  by  modules  and  what  tasks  can  be  shared 
within  a  central  processing  unit,  and  when  functionality  that  had  previously  been 
accomplished  by  multiple  modules  can  be  better  accomplished  in  the  next  iteration  by 
a  single,  integrated  box.  Under  MOSA,  prime  contractors  will  make  those  decisions. 
However,  this  systems  integration  task  will  require  especially  close  collaboration 
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FIGURE  1.  SUMMARY  OF  FIRMS'  ACTIVITIES  UNDER 
THE  MOSA  BUSINESS  MODEL 
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between  the  prime  contractors  and  their  customers,  because  changing  the  boundaries 
of  modules  will  complicate  the  customers’  maintenance  and  upgrade  plans;  modules 
will  no  longer  be  as  interchangeable.  As  a  result,  MOSA  will  emphasize  the  interface 
between  primes  and  their  military  customers. 

Finally,  to  supply  a  given  number  of  systems,  prime  contractors  will  make  more 
separate  offers  to  their  customers  than  they  do  under  the  traditional  acquisition 
approach.  Each  successful  bid  will  win  a  smaller  batch  of  production,  meaning  that  a 
string  of  successes  would  be  required  to  yield  the  same  production  run  as  a  single  win 
would  enable  under  the  traditional  system.  Furthermore,  because  MOSA  facilitates 
competition  and  each  individual  firm  should  expect  to  lose  a  higher  proportion  of  the 
competitions  that  it  enters,  prime  contractors  will  each  need  to  bid  on  a  broader  array 
of  systems  to  maintain  their  workload.5  Bidding  will  have  to  become  a  more  routine 
business  practice,  and  perhaps  well-defined  interfaces  and  modular  product  designs 
will  allow  prime  contractors  to  simplify  the  descriptions  of  what  they  are  trying  to 
sell  to  their  customers.  Moreover,  if  only  a  few  modules  of  a  system  are  changed  from 
one  generation  to  the  next,  a  substantial  fraction  of  a  company’s  offer  (especially  an 
incumbent  producer’s  offer)  may  entail  reuse  of  part  of  the  language  of  the  previous 
iteration  of  competition. 

Most  importantly,  the  content  of  companies’  offers  will  change  under  MOSA. 
Instead  of  explaining  to  the  customer  how  the  firm  plans  to  develop  a  product  to 
meet  the  customer’s  relatively  detailed  specifications,  under  MOSA  a  bid  will  offer 
a  detailed  description  of  the  performance  characteristics  of  a  known  product.  Under 
the  traditional  system,  the  goal  of  a  bid  is  to  convince  the  customer  that  the  firm  is 
likely  to  be  able  to  develop  a  system  at  a  reasonable  cost  and  on  schedule.  Under 
MOSA,  the  prime  contractor  need  not  explain  the  inner  workings  of  the  product  in 
great  detail,  and  at  least  some  technical  characteristics  of  the  system  will  be  described 
by  references  to  widely  known  open  interface  standards.  The  much  simpler  goal  of  a 
MOSA  bid  will  be  to  explain  the  features  of  a  product  and  how  it  meets  needs  set  out 
in  roadmaps. 

Figure  1  summarizes  the  cycle  of  activities  under  the  MOSA  business  model. 


THE  MOSA  BUSINESS  ENVIRONMENT: 

WHAT  THE  GOVERNMENT  MUST  DO 

The  MOSA  will  require  a  substantial  change  in  the  military  acquisition 
organizations’  culture  and  activities.  Specifically,  the  buyer  will  need  to  learn  to  trust 
competition  to  control  prices  and  profits,  replacing  the  current  system’s  direct  audits 
of  program  accounts.  The  buyer  will  also  need  different  kinds  of  technical  knowledge 
than  it  currently  relies  on  to  write  specifications  under  the  traditional  acquisition 
system.  There  is  little  reason  to  believe  that  the  American  government  can  make  these 
changes  in  its  acquisition  processes  for  large  systems  (McNaugher,  1989;  Schooner, 
2003),  but  the  MOSA  is  more  likely  to  work  for  avionics  and  other  subsystems 
contracts,  especially  if  the  equipment  is  purchased  in  small  batches. 
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To  allow  firms  to  amortize  the  development  costs  of  their  many  products  that  do 
not  win  competitions  for  procurement  contracts,  the  government  will  have  to  excise 
the  profit  limits  from  the  Federal  Acquisition  Regulations.  In  essence,  firms  need  to 
figure  the  development  costs  from  their  losing  bids  into  their  calculation  of  overhead 
costs  on  the  production  contracts  that  they  actually  win.  The  result  will  be  that 
procurement  prices  will  be  much  higher  than  the  development  and  production  costs 
of  the  particular  equipment  being  purchased  on  a  particular  contract,  making  that 
contract  seem  tremendously  profitable  if  it  were  examined  under  traditional  cost  and 
pricing  rules. 

Under  the  traditional  acquisition  model,  the  government  faces  tremendous  political 
pressure  to  unilaterally  renegotiate  contracts  that  seem  “too  profitable”:  buyers 
informally  impose  profit  limitations  even  on  fixed-price  contracts  (Rogerson,  1994). 
If  government  auditors  ask  for  too  much  product-specific  cost  data,  the  buyers  will 
face  political  and  cultural  pressure  to  drive  prices  down.  The  buyer  should  not  ask 
questions  to  which  it  cannot  afford  to  know  the  answers.  The  MOSA  contracts  will  be 
fixed-price  with  a  different  cost  structure  than  under  the  current  business  model. 


Firms  will  only  have  an  intentive  to  invest  in  risky, 
innovative  researth  if  the  buyer  allows  them  to 
reroup  their  full  portfolio  of  tosts. 


Firms  will  only  have  an  incentive  to  invest  in  risky,  innovative  research  if  the  buyer 
allows  them  to  recoup  their  full  portfolio  of  costs.  For  those  products  acquired  under 
the  MOSA  business  model,  the  government’s  interest  in  ensuring  affordability  will  be 
maintained  by  competition.  Furthermore,  because  the  government  will  only  buy  small 
batches  at  a  time,  the  buyer  will  not  waste  too  much  money  by  overpaying  on  any 
particular  contract,  if  for  some  reason  competitive  pressures  temporarily  fail  to  limit 
the  profit  margin  to  a  reasonable  level.  Any  purchases  on  which  the  buyer  accidentally 
overpays  will  be  the  contracts  most  likely  to  attract  competitors  for  the  next  round  of 
competition,  driving  the  price  back  down. 

The  other  big  change  in  government  activity  under  MOSA  reinforces  the  recent 
trend  away  from  issuing  detailed  specifications  of  technical  requirements.  Under 
MOSA,  the  buyer  will  simply  solicit  innovative  proposals  from  industry,  based  on 
jointly  developed  roadmaps  that  cover  broad  areas  of  technology. 

But  the  attenuated  government  role  in  technology  management  will  not  absolve 
buyers  of  all  responsibility  for  technical  understanding  of  military  systems.  Buyers 
will  need  two  kinds  of  complex  technical  knowledge.  First,  government  buyers 
will  need  more  technical  skill  to  compare  offers  and  decide  best  value.  Different 
companies  may  not  offer  products  with  the  same  features.  For  example,  one  bid 
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may  offer  exactly  the  same  product  that  the  government  purchased  in  the  previous 
iteration,  perhaps  at  a  lower  price,  while  the  competing  bid  may  offer  a  new,  upgraded 
module  with  extra  functionality,  presumably  at  a  somewhat  higher  price.  Program 
managers  will  need  more  discretion  in  their  source  selection  decisions  than  is  allowed 
by  current  practices  to  weigh  the  value  of  contractor-led  innovations. 

To  earn  that  discretion,  acquisition  officials  will  need  the  technical  capacity  to 
do  more  than  compare  proposals  to  the  specification  or  statement  of  objectives  in 
the  request  for  proposals.  It  may  be  hard  for  the  operational  side  of  the  military 
to  delegate  important  choices  about  technical  performance  to  civil  servants  and 
uniformed  military  acquisition  officers.  Military  leaders  already  chafe  at  decisions  by 
politicians  that  trade  off  performance  against  cost,  but  they  accept  that  determining 
the  defense  budget  is  a  fundamentally  political  decision,  and  military  professionals 
respect  civilian  control  in  the  United  States.  Un-elected  acquisition  officials,  however, 
may  be  more  vulnerable  to  criticism  and  pressure  from  warfighters,  if  they  choose  not 
to  buy  the  most  advanced  technology  available.  Acquisition  is  an  inherently  political 
as  well  as  technical  process  (Dombrowski  &  Gholz,  2006),  and  that  fact  constrains 
the  government’s  ability  to  implement  MOSA. 

The  government  will  also  retain  an  important  role  in  helping  to  set  the  open 
interface  standards — a  second  core  technical  capability.  With  defense  firms  increasing 
their  strength  in  technology  management,  they  naturally  will  have  more  technical 
advice  to  offer  on  the  open  interface  standards.  While  the  government  should 
pay  attention  to  firms’  good  advice,  it  also  needs  an  accountability  mechanism  to 
guard  against  contractors’  natural — perhaps  even  subconscious — attempts  to  seek 
competitive  advantage  and  higher  profitability  by  steering  the  evolution  of  the 
standards  definitions  in  favor  of  particular  technologies. 

Furthermore,  the  organization  empowered  to  set  modular  interface  standards  will 
need  its  own  acquisition  budget.  When  it  determines  that  a  technical  standard  should 
change,  presumably  based  on  an  innovation  created  by  a  single  firm,  that  firm’s 
proprietary  technology  will  have  to  transfer  into  the  public  domain.6  The  standards 
body  will  need  to  pay  to  purchase  that  intellectual  property.  More  than  just  buying 
a  new  system,  the  customer  in  this  case  would  be  buying  part  of  the  competitive 
advantage  that  the  firm  expected  to  enjoy  in  future  rounds  of  competition. 


CONCLUSION:  IS  MOSA  A  GOOD  IDEA? 

The  idea  of  using  a  modular  open  systems  approach  to  buy  military  avionics  is 
relatively  new,  so  at  this  point  assessing  its  benefits  and  costs  is  a  fairly  speculative 
exercise.  It  would  certainly  be  easy  to  oversell  MOSA,  and  overselling  is  often  an 
important  part  of  policy  advocacy.  Convincing  politicians,  military  leaders,  and 
the  acquisition  bureaucracy  to  sign  on  to  a  new  approach  will  require  considerable 
leadership  and  salesmanship.  But  real  analysis  should  support  the  public  approach. 

Work  on  MOSA  for  military  avionics  began  in  the  engineering  community,  where 
many  of  its  technical  advantages  “just  made  sense.”  Digging  deeper  into  the  technical 
issues  raised  important  business  issues:  how  will  defense  companies  operate  under 
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the  MOSA,  and  can  they  be  induced  to  support  MOSA  by  offering  them  a  reasonable 
chance  to  earn  profits?  This  description  of  a  possible  MOSA  business  model  suggests 
that  businesses  can  adapt  to  MOSA,  at  least  for  some  military  avionics  projects.  But 
getting  past  the  business  problems  only  reveals  political  and  organizational  questions 
that  need  to  be  answered,  too,  before  significant  acquisitions  will  make  sense  under 
the  new  approach.  Moreover,  MOSA  will  surely  involve  a  good  deal  of  painful 
reorganization  and  overhead  investment  in  creating  the  standards  body — costs  that 
need  to  be  considered  carefully  compared  to  the  limited  volume  of  MOSA  projects 
that  will  be  available  for  bidding  even  under  a  fully  implemented  MOSA  acquisition 
system. 

The  MOSA  offers  some  clear  advantages  for  both  the  DoD  and  the  defense 
industry.  Systems  designed  for  modular  maintenance  and  upgrades  should  be  able 
to  resolve  some  of  the  obsolescence  problems  of  today’s  equipment,  and  technology 
refresh  opportunities  should  facilitate  spiral  development,  allowing  equipment  to 
better  serve  warfighters’  needs.  The  frequent  competitions  to  sell  small  batches  of 
modules  and  systems  should  also  allow  sensible,  flexible  decision  making  to  trade 
off  maintenance  and  acquisition  spending.  Firms  will  be  eager  to  take  control  of  their 
technology  management  decision-making  and  to  rely  on  their  own  strategic  decisions 
about  investment  priorities. 

On  the  other  hand,  MOSA  must  overcome  some  real  limits.  The  appeal  of  open 
standards  and  the  analogy  to  the  world  of  commercial  information  technology  are 
frequently  used  to  support  casual  claims  about  gains  in  interoperability.  While 
increasing  interoperability  is  surely  an  extremely  important  goal  for  military 
acquisition  organizations  in  the  new  era  of  network-centric  warfare,  it  is  easy  to 
exaggerate  the  interoperability  benefits  of  MOSA.  Not  every  commercial  IT  product 
really  “plugs  and  plays”;  hard  work  is  still  required,  not  only  to  define  the  interface 
standards  but  to  maintain  them  in  the  face  of  technological  change  and  mission  creep. 


It  is  easy  to  exaggerate  the  interoperability  benefits 
of  MOSA.  Not  every  (ommerdal  IT  produtt  really 
"plugs  and  plays." 


The  organization  in  charge  of  maintaining  the  open  architecture  standards, 
supported  by  government  project  managers,  will  need  to  decide  the  extent  to  which 
the  detailed  designs  for  each  new  product  purchased  by  the  military  will  pass  into 
the  open  architecture.  If  too  much  technology  remains  proprietary,  then  the  MOSA 
business  model  will  not  reduce  costs  to  the  extent  that  it  should,  i.e.,  competitive 
firms  will  still  have  to  re-invent  the  wheel  over  and  over  again.  But  each  innovation 
that  changes  the  standards  definitions  must  be  fully  paid  for  at  the  time  that  it 
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is  shared  with  the  rest  of  the  defense  industry,  meaning  that  incorporating  new 
technologies  into  the  standards  definitions  will  be  expensive.  Moreover,  changing 
the  standards  too  frequently  will  set  back  the  gains  in  interoperability  that  MOSA 
is  intended  to  bring  and  will  attenuate  the  learning-by-doing  benefits  that  designers 
would  otherwise  gain  by  repeatedly  using  the  same  architecture. 

For  other  reasons,  too,  the  cost  advantage  of  MOSA  may  not  be  large.  Buyers 
will  still  insist  on  dictating  the  technology  trajectory,  so  MOSA  will  not  allow  for 
innovation  to  move  in  its  most  “natural”  direction.  Appeals  to  national  security  and 
the  needs  of  warfighters  carry  a  tremendous  amount  of  weight,  especially  in  times  of 
war,  and  technology  experts  and  business  strategists  will  struggle  to  make  their  views 
heard  in  the  roadmapping  process. 

Furthermore,  MOSA  requires  a  delicate  balance  between  the  costs  and  benefits  of 
competition.  Development  cost  savings  under  MOSA  come  from  getting  modules 
into  multiple  systems:  winning  bidders  need  to  succeed  repeatedly.  At  the  same 
time,  the  margin  between  price  and  cost  is  only  controlled  by  competition,  and 
each  competition  to  sell  a  batch  of  equipment  will  have  many  losers.  Somehow,  the 
acquisition  system  needs  to  pay  the  development  costs  of  all  of  those  losing  bidders 
to  keep  them  in  the  defense  business.  The  more  firms  that  bid  on  each  increment  of 
technology,  the  more  total  development  investment  that  has  to  be  spread  across  the 
production  runs  of  successful  bids.  If  too  many  bidders  are  attracted  to  the  MOSA 
market,  MOSA  could  actually  increase  system-wide  costs. 


Ultimately,  MOSA  may  be  a  workable  way  to  solve  a  number 
of  tethnital  and  business  protess  problems  for 
the  atquisifion  of  military  avionits. 


Finally,  firms  should  not  necessarily  trust  the  government.  First,  the  ultimate 
buyers  of  military  systems,  Congressional  representatives,  may  not  allow  firms  to 
set  prices  high  enough  to  cover  their  total  costs  of  bidding  for  MOSA  contracts. 
Politicians  are  likely  to  “renegotiate”  profit  when  it  seems  too  high  on  an  individual 
product.  Contracts  for  subsystems  and  small  modules  are  likely  to  be  “too  far  in  the 
weeds”  for  politicians  to  notice,  but  to  make  a  big  difference  in  interoperability,  spiral 
development,  and  cost  savings,  MOSA  will  have  to  apply  to  some  relatively  visible 
products,  too.  The  only  hope  is  that  sales  can  be  packaged  in  small  enough  batches 
not  to  attract  politicians’  attention  to  the  profit  margins.  But  the  small  batches  are  a 
double-edged  sword  for  firms  trying  to  manage  their  portfolio  of  risk:  the  smaller 
their  guaranteed  production  runs  from  each  contract  win,  the  higher  the  risk  the  firms 
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will  have  to  bear.  For  MOSA  to  work,  batch  sizes  need  to  be  set  pragmatically  (not 
too  small),  and  that  will  require  a  good  political  solution. 

Ultimately,  MOSA  may  be  a  workable  way  to  solve  a  number  of  technical  and 
business  process  problems  for  the  acquisition  of  military  avionics.  But  “workable” 
does  not  mean  that  MOSA  is  a  good  way  to  solve  those  problems.  Unfortunately,  by 
its  very  nature,  it  will  probably  not  be  possible  to  implement  small-scale  “proof  of 
concept”  tests  of  the  modular  open  system  approach:  it  inherently  relies  on  spreading 
risk  and  investment  cost  across  a  broad  array  of  projects  all  at  the  same  time.  The 
DoD  Open  Systems  Joint  Task  Force  is  working  to  implement  MOSA,  and  recent 
reforms  of  the  defense  acquisition  process  make  MOSA  a  default  approach  for 
some  systems.  With  that  in  mind,  defense  industry  executives  and  the  government 
acquisition  workforce  need  to  understand  the  business  issues,  as  well  as  the  technical 
and  political  ones,  in  the  Modular  Open  Systems  Approach. 
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ENDNOTES 


1.  A  broad  selection  of  this  research  can  be  accessed  on  the  web  site  of  the  DoD’s 
Open  Systems  Joint  Task  Force  at  http://www.acq.osd.miFosjtf/index.html. 

2.  The  MOSA  also  faces  political  barriers.  While  the  Department  of  Defense  can 
(and  already  does)  include  a  preference  for  MOSA  in  its  acquisition  regulations, 
and  Congressional  leaders  in  principle  support  the  idea  of  efficient  acquisition 
and  can  understand  the  arguments  that  have  piqued  the  military  customers’ 
interest  in  MOSA,  political  leaders  nevertheless  have  good  reasons  to  perpetuate 
the  traditional  style  of  acquisition  for  the  vast  majority  of  defense  projects.  The 
traditional  acquisition  system  did  not  evolve  by  accident.  The  MOSA  advocates 
should  conduct  a  separate  analysis  of  the  political  case  for  MOSA  and  should 
develop  a  political  strategy  to  broaden  MOSA’s  application  in  parallel  with  their 
technical  analyses  and  the  business  analysis  reported  in  this  article. 

3.  There  is  no  obvious  reason  why  MOSA  would  require  any  shift  in  the  small 
business  set-asides  in  defense  contracting.  Prime  contractors  will  simply  continue 
to  include  small  business  content  as  one  of  the  factors  to  optimize  in  their  trade 
studies  during  system  development.  Meanwhile,  the  buyers  will  continue  to 
include  small  business  content  as  one  of  the  desirable  factors  that  they  weigh 

in  determining  whether  to  pay  the  asking  price  for  a  system  offered  by  a  prime 
contractor.  As  a  result  of  this  continuity,  the  overall  shift  to  the  MOSA  business 
model  is  unlikely  to  require  substantial  changes  to  the  business  models  followed 
by  defense-oriented  small  businesses. 

4.  Some  products  may  be  too  complex  and  some  systems  may  require  too  much 
up-front  investment  for  firms  to  bear  the  costs  alone.  For  those  systems,  the 
government  and  the  defense  industry  may  continue  to  use  the  existing  weapons 
acquisition  model. 

5.  Assuming  that  the  introduction  of  MOSA  does  not  increase  the  total  amount  of 
equipment  that  the  military  demands,  any  new  suppliers  that  are  drawn  into  the 
military  market  by  MOSA  will  have  to  take  work  from  established  suppliers. 
Presumably,  many  of  the  new  entrants  that  MOSA  advocates  hope  to  bring 
into  the  defense  industry  will  offer  modules  rather  than  systems,  meaning  that 
they  will  compete  more  directly  with  Tier  2  subcontractors.  But  established 
defense  firms  at  all  levels  of  the  industry  should  wonder  if  one  result  of  the 
new  acquisition  approach  would  be  to  shrink  the  per  capita  market  size,  hence 
shrinking  their  expected  business  volume,  revenue,  and  employment  levels 
(thought  not  necessarily  their  rate  of  return). 

6.  “Public”  in  this  context  does  not  mean  freely  available  to  anyone.  It  means  open 
to  firms  allowed  access  to  classified  technical  standards  available  for  use  by  firms 
in  the  defense  industry. 
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DECISION  SUPPORT  FOR 
BEST  PRACTICES: 
LESSONS  LEARNED  ON 
BRIDGING  THE  GAP 
BETWEEN  RESEARCH  AND 
APPLIED  PRACTICE 


Raimund  L.  Feldmann,  Forrest  Shull,  and  Michele  A.  Shaw 


Today,  everyone  is  looking  at  best  practices  for  developing  a  system  or  making 
the  right  choice  in  acquiring  system  components.  If  the  right  best  practices 
are  applied,  they  help  to  avoid  common  problems  and  improve  quality,  cost, 
or  both.  However,  finding  and  selecting  an  appropriate  best  practice  is  not 
always  an  easy  endeavor.  In  most  cases  guidance,  based  on  sound  experience, 
is  missing;  often  the  best  practice  is  too  new,  still  under  study,  or  the  existing 
experiences  do  not  fit  the  user's  context.  This  article  reports  on  a  program 
that  tries  to  bridge  the  gap  between  rigorous  empirical  research  and  practical 
needs  for  guiding  practitioners  in  selecting  appropriate  best  practices. 


Many  program  managers  would  agree  that  using  time-tested  “Best  Practices” 
can  help  to  avoid  common  problems  and  increase  the  quality  of  a  system, 
reduce  development  cost,  or  both.  For  instance,  in  a  short  survey  at  the  2004 
Conference  on  the  Acquisition  of  Software-Intensive  Systems,  48  senior  systems 
and  software  managers  supported  the  use  of  Best  Practices.  However,  the  same 
survey  indicated  that  it  is  hard  to  find  such  Best  Practices.  The  survey  identified  the 
following  reasons  for  this  problem: 

■  Best  practices  often  do  not  exist  (i.e.,  they  have  not  been  publicly  documented), 
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■  People  do  not  know  of  a  certain  best  practice,  or 

■  Best  practices  are  not  easily  accessible  (i.e.,  there  is  no  central  place  to  look  for  best 
practices). 

The  last  point  matches  a  more  general  study  by  the  Delphi  Group  in  which  more 
than  65  percent  of  the  interviewees  agreed  that  finding  the  right  information  to  do 
their  job  is  difficult  (Delphi,  2002). 

Further  research  conducted  by  the  U.S.  Department  of  Defense  (DoD)  concluded 
that  barriers  for  the  adoption  of  best  practices  included: 

■  the  lack  of  selection  criteria  among  practices  within  cost-constrained  programs, 

■  the  lack  of  confidence  in  the  value  of  such  practices  by  the  program  offices,  and 

■  the  inability  to  relate  practices  to  the  risks  and  issues  programs  were  facing. 

In  summary,  recognizing  good  practices  and  disseminating  them  to  the  workforce 
seems  to  be  a  key  issue.  To  address  these  issues  the  DoD  Acquisition  Best  Practices 
Clearinghouse  (BPCh)  program,  sponsored  by  several  offices  of  the  DoD  (  DS,  ARA, 
National  Information  Infrastructure  [Nil],  and  Defense  Procurement  &  Acquisition 
Policy  [DPAP]),  was  initiated  in  2003  (Dangle,  Dwinnell,  Hickok  &  Turner,  2005). 

The  Fraunhofer  Center  for  Experimental  Software  Engineering,  Maryland 
(FC-MD)  was  chosen  to  develop  the  initial  “proof  of  concept”  for  a  system  to 
document,  evaluate,  and  disseminate  Best  Practices.  In  collaboration  with  other 
organizations  within  the  DoD  and  industry  (including  Northrop  Grumman  IT,  the 
Computer  Sciences  Corporation  [CSC],  and  the  Systems  and  Software  Consortium 
[SSCI]),  a  prototype  system  has  been  built  and  piloted.  It  is  currently  operated  and 
hosted  by  the  Defense  Acquisition  University  (DAU). 


THE  VISION  FOR  APPLYING  BEST  PRACTICES 

The  DoD  vision  for  the  BPCh  initiative  is  to  provide  more  than  just  a  list  of  Best 
Practices.  It  is  to  provide  an  integrated  set  of  processes,  tools,  and  resources  which 
will  enable  information  seekers  to  identify  emerging  or  well-proven  practices  that 
have  been  implemented  and  proven  effective.  Practices  in  the  BPCh  serve  as  an 
information  resource  to  individuals  looking  for  ideas  on  how  to  improve  quality  and 
become  more  effective  in  their  job.  Clearly,  the  vision  of  the  BPCh  is  not  to  create 
another  “data  cemetery,”  but  to  develop  an  information-sharing  network  around  the 
BPCh  repository  which  will  foster  relationships  between  individuals  within  DoD 
and  also  partnerships  between  DoD  and  industry  leaders.  The  following  types  of 
questions  illustrate  usage  examples: 
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■  “I  just  heard  about  accelerated  life  testing.  Where  can  I  find  out  if  it’s  useful  or  just 
hype?” 

■  “They’ve  just  shortened  my  testing  schedule  by  30  percent.  Are  there  any  practices 
that  can  help  me  better  handle  that  kind  of  schedule  compression?” 

■  “I  want  to  add  inspections  to  my  quality  process.  Is  it  worth  the  cost  and  if  so,  what’s 
a  good  first  step?  Is  there  someone  I  can  contact  in  case  of  any  difficulties?” 

■  “five  taken  over  an  acquisition  program  just  before  Critical  Design  Review  (CDR). 
What  practices  should  I  look  for  in  my  contractors?” 

■  “I’m  in  charge  of  defining  a  training  course  as  part  of  the  continuing  education 
program  for  quality  improvements.  What  are  state-of-the-art  or  emerging  practices 
that  should  be  addressed?” 

The  BPCh  has  been  designed  with  the  understanding  that  a  single  practice  can 
never  be  a  “silver  bullet”  for  each  and  every  project/program.  This  is  because  some 
practices  may  only  be  useful  or  beneficial  in  certain  contexts  while  failing  to  produce 
the  desired  results  in  others.  For  example,  practices  that  are  absolutely  necessary 
for  large,  mission  critical  projects  may  be  too  heavyweight  for  rapid  prototyping  or 
Web  application  development.  Practices  that  work  well  when  the  development  team 
is  located  in  the  same  room  may  not  always  scale  well  when  the  team  is  distributed 
across  the  country. 

Clearly,  there  exists  no  one  “best”  answer.  Practices  that  are  best  for  one  user  might 
not  be  best  for  the  next.  Therefore,  the  BPCh  tool  responds  to  user  queries  with  a  list 
of  practices  rated  by  how  well  they  fit  the  project  characteristics  of  the  user  making 
the  query.  The  presented  selection  is  compiled  using  the  experience  other  users  have 
had  implementing  the  practice  in  a  similar  context.  High-quality  evidence  about  a 
practice  is  collected  and  reported  with  any  necessary  caveats,  so  that  information 
seekers  have  a  sound  basis  for  making  up  their  own  minds  given  their  needs. 


APPLYING  TECHNOLOGY  TO  DELIVER  BEST  PRACTICES 

To  develop  the  BPCh  tool,  we  applied  FC-MD’s  EMPEROR  approach  (Experience 
Management  Portal  using  Empirical  Results  as  Organizational  Resources).  This 
approach  makes  use  of  all  kinds  of  available  evidential  data  from  research  and 
industry,  analyzes  and  packages  it,  and  disseminates  it  through  a  Web-based 
Experience  Base. 

The  EMPEROR  is  based  on  the  experience  factory  approach,  developed  by 
Basili,  Caldiera,  and  Rombach  (1994),  which  has  been  successfully  employed  to 
facilitate  organizational  learning  at  NASA  (Basili,  et  al.,  1995),  DaimlerChrysler 
(Schneider  &  Schwinn,  2001),  and  elsewhere  in  North  America,  Europe,  and 
Australia  (Koennecker,  Jeffery,  &  Low,  2000;  Mendonca,  Seaman,  Basili,  &  Kim, 
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2001).  An  experience  factory  provides  a  way  to  analyze  results  based  on  practical 
experience,  and  package  what  is  learned  into  an  Experience  Base  for  new  users  of  the 
organization  to  find  and  apply. 

Since  the  users  of  the  BPCh  come  from  a  wide  variety  of  organizations  and 
programs,  any  Experience  Base  will  have  difficulties  in  addressing  all  user  needs.  To 
mitigate  this  problem,  EMPEROR  is  required  to:  (a)  provide  transparency  to  users,  so 
that  they  can  understand  the  analysis  process  and  the  sources  of  experience  and  make 
up  their  own  minds;  (b)  rate  the  “trustability”  of  each  of  the  used  sources,  so  that 
users  can  judge  the  degree  of  confidence  they  have  in  the  information  provided;  and 
(c)  provide  a  completeness  and  maturity  indicator  of  the  practice  information  taken 
as  a  whole,  that  is,  to  perform  a  self-rating  based  on  how  much  and  what  quality 
evidence  can  be  offered. 


DATA  STRUCTURE  OF  A  BPCH  PRACTICE 

These  sections  describe  how  these  requirements  are  implemented  in  the  case  of 
the  BPCh.  In  the  BPCh,  each  practice  has  one  associated  Practice  Record,  containing 
information  about  the  practice  and  what  is  available  in  the  Clearinghouse,  and  zero  to 
many  Evidence  Profiles,  each  of  which  contains  a  summary  of  a  single  organization’s 
experience  using  the  practice. 

A  Practice  Record  consists  of: 

1 .  A  Practice  Detail  block,  which  contains  information  such  as  the  practice  name,  a 
short  description,  and  the  completeness  and  maturity  indicator  for  the  experience 
package. 

2.  A  Practice  Summary  block,  which  synthesizes  all  available  evidence  data 
and  describes  possible  application  contexts  for  the  practice  based  on  a  set  of 
characterizing  attributes.  This  part  of  the  practice  record  thereby  allows  different 
users  (i.e.,  organizations)  to  make  use  of  the  practice. 

An  Evidence  Profile  contains  an  example  or  report  of  some  type  of  program  that 
has  used  this  practice,  how  they  applied  it,  and  what  results  were  obtained.  Each 
Evidence  Profile  contains  the  same  set  of  context  and  result  fields  as  the  Practice 
Summary  block,  except  that  the  information  recorded  in  each  field  will  describe  only 
what  has  been  observed  in  the  given  context  of  the  particular  piece  of  evidence.  In 
addition,  the  data  structure  of  an  Evidence  Profile  contains  a  field  for  documenting  its 
classification  of  the  trustability. 

TRUSTABILITY  OF  A  SINGLE  SOURCE  OF  EVIDENCE 

A  20-point  scale  rates  the  trustability  of  each  Evidence  Profile.  A  rating  of  1 
indicates  an  anecdotal  or  informal  experience;  a  rating  of  20  indicates  that  the  results 
of  applying  the  practice  are  rigorously  measured  and  substantiated.  Points  are  based 
on  the  following  four  dimensions: 
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■  how  the  practice  was  applied,  ranging  from  a  single  pilot  study  to  use  on  multiple 
real  projects; 

■  how  the  results  were  measured,  ranging  from  an  educated  guess  to  a  rigorous 
measurement  program; 

■  how  the  evidence  was  reported,  ranging  from  an  informal  anecdote  to  a  peer- 
reviewed  publication;  and 

■  who  reported  the  evidence,  ranging  from  a  second-hand  report  to  someone  directly 
involved  on  the  team. 

More  information  on  the  rating  scale  can  be  found  on  the  BPCh  page  of  the 
Acquisition  Community  Connection  of  DAU  (https://acc.dau.mil/bpch). 


MATURITY  OF  A  PRACTICE  RECORD 

A  4-point  scale  is  used  to  rate  each  Practice  Record  to  quickly  inform  the  user  of 
how  much,  and  what  type  of,  information  is  known  about  the  practice.  As  required  by 
EMPEROR,  this  scale  focuses  on  the  quality  of  the  overall  accumulated  information 
that  is  available  for  a  practice  (i.e.,  the  synthesized  and  packaged  information  in 
the  Practice  Record).  Based  on  the  available  information  we  describe  the  practice 
maturity  as: 

■  No  status  assigned/initial  entry:  A  new  Practice  Record  is  initially  entered  into  the 
BPCh  when  it  is  nominated  by  our  experts  and/or  user  communities.  Typically  at 
this  time,  only  some  of  the  fields  in  the  Practice  Detail  block  are  filled  in  and  no 
Evidence  Profiles  are  available. 

■  Bronze  status/Awareness  raised:  As  soon  as  any  evidence  becomes  available  (i.e., 
an  Evidence  Profile  has  been  linked  to  the  Practice  Record),  the  status  is  set  to 
Bronze  Level.  For  users,  the  Bronze  Level  status  indicates  that  the  practice  has  been 
nominated  by  our  experts  and  user  communities,  and  received  a  preliminary  check 
for  applicability. 

■  Silver  status/Evaluation  performed:  When  a  sufficient  set  of  Evidence  Profiles  is 
available,  the  BPCh  experts  will  fill  in  the  Practice  Summary  block  and  the  status  is 
set  to  Silver  Level.  For  users,  the  Silver  Level  status  indicates  that  the  practice  has 
been  selected  as  promising  enough  to  commission  experts  in  the  area  to  summarize 
key  information.  Users  can  see  at  a  glance  what  they  should  know. 

■  Gold  status/Continuously  maintained:  When  the  summary  has  been  further  evaluated 
(i.e.,  vetted)  by  experts  from  industry,  academia,  and  government,  the  status  is  set 
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to  Gold  Level.  For  users,  the  Gold  Level  status  indicates  that  the  practice  has  been 
through  a  rigorous  analysis  by  a  committee  of  experts  in  the  practice  itself  as  well  as 
by  user  representatives.  Information  on  Gold  Level  practices  contains  the  best  and 
widest-ranging  experiences  we  can  find. 


CONTENT  STATUS  OF  THE  BPCH 

We  have  been  piloting  BPCh  processes  and  tools  by  seeding  initial  content. 

At  this  point  the  BPCh  contains  5 1  practices  at  all  levels  of  maturity.  Practices  that 
have  progressed  to  Gold  Level  are  those,  like  inspection/technical  review,  which  have 
a  long  history  of  published  industrial  experience. 

Many  practices  of  interest  in  the  area  of  systems  and  software  acquisition  have  few 
documented  sources  of  evidence  or  experience.  Therefore,  we  are  testing  different 
processes  for  eliciting  information  from  the  workforce. 

Based  on  the  recommendations  of  our  User  Advisory  Group,  the  following  types 
of  practices  are  currently  our  top-priority  areas  for  additional  content: 

■  Earned  Value  Management, 

■  Risk  Management, 

■  Information  Assurance,  and 

■  Spiral  Development  Process. 

We  hope  that  visitors  to  the  BPCh  tool  will  try  out  the  offered  features  for 
providing  short  stories  about  their  own  experience  with  practices  in  these  (or  any 
other)  areas.  We  encourage  you  to  provide  feedback  as  to  whether  you  agree  or 
disagree  with  the  existing  experiences  that  have  been  entered,  or  thoughts  on  our 
BPCh  tool  in  general. 


LESSONS  LEARNED 

Based  on  our  experience  with  the  BPCh  program  and  other  knowledge- 
management  projects,  we  can  formulate  some  observations  which  make  useful  rules 
of  thumb  for  good  practices  to  build  such  systems.  The  BPCh  program  has  been 
organized  along  three  parallel  (but  interconnected)  tracks,  which  reflects  our  first 
lesson  learned. 

LESSON  1 :  PROCEED  IN  MULTIPLE  DIRECTIONS  SIMULTANEOUSLY 

Progress  in  building  a  knowledge  repository  needs  to  proceed  in  multiple 
dimensions  simultaneously:  content  collection,  tool  development,  and  outreach. 
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Although  there  is  often  a  temptation  to  view  these  as  tasks  that  can  be  done 
sequentially  (e.g.,  first  the  tool  will  be  built,  then  populated,  and  then  it  will 
be  advertised  to  users),  we  have  found  this  to  be  an  overly  simplistic  view  that 
diminishes  the  chance  of  project  success.  Constructing  the  tool  prior  to  collecting 
actual  content  and  getting  users’  feedback  almost  ensures  that  important  user  needs 
will  be  discovered  late  and  will  require  much  more  effort  to  implement.  Populating 
the  content  without  getting  user  feedback  leads  to  a  high  likelihood  that  the  content 
will  not  really  address  user  needs.  More  importantly,  content  needs  to  come  from  the 
user  community,  if  the  repository  is  to  have  a  long-term  life.  We  have  found  that  for 
the  research  team  to  generate  substantial  amounts  of  content  is  a  time-consuming  way 
of  recreating  what  many  users  already  have  at  their  finger  tips.  Finally,  engaging  in 
outreach  and  building  excitement  in  the  community  of  potential  users  runs  the  risk  of 
all  prototyping  efforts:  When  told  how  anything  is  possible  in  the  final  system,  users 
often  come  up  with  many  wish  list  features  that  are  not  really  linked  to  their  everyday 
needs.  Moreover,  users  often  get  frustrated  with  the  slow  pace  of  progress  when  the 
system  actually  has  to  be  implemented,  and  lose  interest  before  the  system  is  fielded. 


Constructing  the  tool  prior  to  tolletting  actual  content  and 
getting  users'  feedback  almost  ensures  that  important  user 
needs  will  be  discovered  late  and  will  require  much  more 
effort  to  implement. 


To  avoid  these  problems,  we  have  adopted  an  incremental  approach,  with  content 
and  tool  development  going  on  simultaneously  and  outreach  activities  to  the  user 
community  (such  as  booths  at  major  conferences,  or  specific  User  Advisory  Group 
meetings)  planned  at  major  milestones.  Although  this  sometimes  stretches  resources 
a  bit  thin,  we  feel  this  approach  has  enabled  us  to  engage  periodically  with  the  user 
community,  show  them  progress  since  the  last  iteration,  and  get  feedback  on  ever 
more  mature  versions  of  the  system,  with  an  initial  body  of  content. 

LESSON  2:  MAINTAIN  A  CONTINUOUS  STREAM  OF  FUNDING 

Because  of  the  interconnected  nature  of  all  the  tasks  listed  above,  having  a  stable 
funding  stream  is  crucial.  Requiring  the  team  to  take  a  hiatus  from  the  project  after 
a  release  is  delivered  leads  to  lost  opportunities  for  user  involvement  (users  find  it 
hard  to  match  their  schedule  to  the  development  team’s),  leads  to  new  content  ideas 
that  miss  getting  followed  up  on,  may  result  in  the  loss  of  expertise  if  experienced 
personnel  resources  are  in  transition  to  other  projects  during  the  hiatus,  increases  the 
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personnel  learning  curve  encountered  at  restarts,  and  may  result  in  flagging  interest  in 
the  user  community  since  momentum  generated  during  outreach  is  lost. 

LESSON  3:  RECOGNIZE  THE  RELATIVE  MERITS  OF  CONTENT 

Our  most  important  lesson  learned  is  a  direct  implication  of  the  BPCh  vision: 

There  is  no  such  thing  as  a  “Best  Practice.”  Or,  to  say  it  more  diplomatically:  No 
practice  will  be  “best”  for  every  project.  Practices  that  are  absolutely  necessary  for 
large,  mission-critical  projects  may  be  too  heavyweight  for  rapid  prototyping  or  Web 
application  development. 

The  implications  of  this  lesson  are  many.  Perhaps  the  most  important  is  related  to 
the  tone  of  the  recommendations  that  users  find:  Rather  than  arguing  as  an  expert  that 
readers  should  be  following  a  given  practice,  or  else  they  are  doing  something  wrong, 
practices  should  be  recommended  to  readers  on  the  basis  that  projects  of  certain 
type(s)  have  found  it  useful.  That  is,  rather  than  presenting  a  foregone  conclusion  to 
users,  the  system  should  aim  at  respecting  users’  intelligence  enough  to  enable  them 
to  draw  their  own  conclusion,  providing  sufficient  evidence  as  necessary  for  those 
decisions  to  be  sound  ones. 

LESSON  4:  UNDERSTAND  THE  LIFE  CYCLE  OF  BEST  PRACTICES 

Practices  (and  practice  information)  are  not  static  and  have  a  real  life  cycle.  Major 
paradigm  shifts  in  the  software  development  world  can  have  an  impact  on  which 
practices  are  recommended.  The  practices  that  seemed  to  be  good  fits  for  most 
projects,  when  a  waterfall  life  cycle  was  the  most  common  approach  to  software 
development,  are  not  all  equally  applicable  at  the  current  time,  when  iterative,  spiral, 
and  even  agile  approaches  are  probably  more  representative  of  the  state-of-the-art 
practice. 

Our  recommendations  regarding  a  structured  life  cycle  for  practice  information 
are: 

1 .  A  knowledge  repository  needs  to  be  continually  evolving  by  accepting 
information  on  topics  of  interest  and  making  it  available  to  users  as  soon  as 
possible.  While  some  quality  checking  is  necessary  to  make  sure  that  incorrect, 
misleading,  or  incomplete  information  is  disseminated  outward,  it  is  better  to 
get  information  to  users  as  it  comes  in,  than  to  wait  and  try  to  create  something 
perfect.  Users  should  be  able  to  see  a  timestamp  on  all  information  so  that  they 
can  see  if  the  experiences  related  are  fresh  and  up  to  date  or  come  from  years 
ago. 

2.  However,  the  desire  to  get  information  out  quickly  should  not  interfere  with  the 
need  for  validation  activities  that  provide  higher  confidence  in  the  information. 
These  additional  levels  of  maturity  should  be  noted,  to  give  users  more 
confidence  in  the  information  they  find,  but  should  not  be  used  as  a  precondition 
for  displaying  content. 
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3.  Content  needs  to  be  retired  when  appropriate.  Practices  may  have  a  natural 
lifespan,  since  the  acquisition  and  development  worlds  continue  to  evolve 
and  change  on  their  own.  Practices  that  were  good  1 0  years  ago  may  not  be 
appropriate  given  today’s  constraints  or  technologies.  To  avoid  users  finding 
obsolete  information  in  the  repository,  reports  need  to  be  generated  periodically 
of  which  practices  have  received  no  updates  or  new  experiences  in  the  longest 
time. 

LESSON  5:  APPLY  AGILE  STRATEGIES  AND  PROTOTYPING 

To  create  the  front  end  of  the  BPCh  tool,  which  helps  users  find  candidate 
practices,  explore  possibilities,  and  get  more  information  on  practices  of  real  interest, 
we  have  found  that  prototyping  and  agile  strategies  are  extremely  valuable  for 
developing  knowledge-management  systems. 


Speak  to  the  users  in  their  language. 
Do  not  expert  them  to  learn  yours. 


Precisely  because  of  the  need  for  parallel  activities  in  different  tracks,  and  the 
number  of  stakeholders  involved  (tool  developers,  content  gathering  team,  end  user 
representatives,  sponsor  representatives),  an  agile  approach  is  extremely  valuable.  The 
implementation  of  the  prototype  BPCh  tool  was  carried  on  in  two-week  increments, 
at  the  end  of  which  a  releasable  version  was  always  available.  At  the  end  of  each 
two-week  period,  a  demonstration  and  planning  meeting  was  held  with  as  many  of  the 
stakeholders  as  could  be  present.  This  approach  was  necessary  to  help  us  coordinate 
and  prioritize  the  evolving  expectations  of  the  users  as  well  as  the  necessary  changes 
that  were  suggested  by  the  content  development  team,  based  on  what  they  were 
finding. 

As  part  of  this  meeting  we  learned  the  following  lesson: 

LESSON  6:  USE  APPROPRIATE  LANGUAGE 

Speak  to  the  users  in  their  language.  Do  not  expect  them  to  learn  yours.  We 
realized  early  on  that  having  the  greatest  possible  content  in  the  BPCh  repository 
would  not  be  of  much  help  if  the  users  cannot  find  it.  To  address  this  we  needed  to 
provide  multiple  paths  to  the  information,  so  that  users  could  select  the  path  that 
made  the  most  sense  to  them.  Some  specific  lessons  learned  here  included: 

1.  Organize  around  common  tasks.  The  best  way  to  reach  users  is  to  organize  the 
contents  of  the  repository  according  to  everyday  activities  that  the  user  performs. 
This  helps  users  see  the  repository  less  as  an  additional  activity  that  they  need  to 
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make  time  for,  and  more  as  a  value-added  to  the  activities  that  already  consume 
their  time.  In  the  case  of  BPCh,  we  added  several  such  perspectives  (i.e.,  indexes 
to  the  content)  based  around  activities  of  importance  to  different  segments  of  the 
user  community  (e.g.,  addressing  CMMI  practice  areas,  constructing  a  systems 
engineering  strategy,  and  referencing  back  to  common  guidebooks). 

2.  Push  as  well  as  pull  information.  Rather  than  always  expecting  users  to  take  time 
to  come  to  browse  the  BPCh  tool,  information  can  be  “pushed”  outward  to  the 
user  on  a  periodic  basis.  For  example,  the  user  could  select  some  practices  of 
special  interest,  and  when  new  experiences  come  in  related  to  these  practices  a 
notification  is  sent  via  e-mail. 

3.  Match  users  to  practices  based  on  context  similarity.  Since  no  practice  will  be 
“best”  for  every  project,  it  is  important  to  match  users  to  practices  using  context 
characteristics.  This  provides  the  users  with  a  pick  list  of  practices  that  may  be 
useful  in  their  particular  situation.  In  addition,  it  may  alert  the  user  to  practices 
that  they  might  not  have  known  about  previously.  For  example,  if  the  user 
selects  a  few  context  variables  that  describe  his/her  context,  then  practices  can 
be  prioritized  and  displayed  according  to  whether  they  have  associated  evidence 
provided  by  users  with  similar  context  information.  This  is  a  way  of  indicating 
that,  even  if  the  practice  does  not  answer  a  specific  search  query,  users  like  the 
current  one  have  found  this  practice  useful  and  it  may  be  something  the  user 
should  know. 

LESSON  7:  DEMONSTRATE  PRACTICAL  EXAMPLES  TO  INTENDED  USER 

To  engage  in  effective  outreach  activities,  aimed  at  building  up  an  interested  and 
active  community  of  users  of  the  BPCh,  we  find  the  following  lesson  of  relevance: 

You  can  not  show  initial  users  an  empty  depository. 

In  line  with  the  idea  that  building  a  tool  like  the  BPCh  needs  to  proceed  on  three 
tracks  in  parallel  (front-end,  content,  and  outreach)  is  the  lesson  that  populating  the 
content  cannot  come  after  the  repository  is  built.  Showing  users  a  fancy  front-end 
without  an  initial  set  of  real  content  may  get  their  interest  for  a  short  time  period, 
but  is  not  an  effective  way  of  building  an  active  user  community.  Users  need  to  see  a 
small  but  representative  set  of  content  which  they  can  respond  to  and  start  generating 
ideas  for  the  next  content  or  tool  release. 

LESSON  8:  UPDATE  CONTENT  AND  FUNCTIONALITY  CONTINUOUSLY 

To  keep  interest  engaged,  when  users  do  check  back  to  the  site  they  need  to  see  that 
updates  have  been  made  since  last  time.  Content  needs  to  be  continuously  updated 
and  refreshed  to  stay  abreast  of  trends. 

If  users  ever  become  convinced  that  the  repository  does  not  get  updated  on  a 
regular  basis,  this  often  spells  the  end  of  their  involvement.  Rather,  they  need  to 
be  motivated  to  come  back  often  enough  to  find  new  things  and  hopefully,  as  they 
progress,  be  motivated  to  submit  responses  and  ideas  of  their  own  showing  emerging 
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trends  and  keeping  the  content  relevant.  Thus,  user  involvement  tends  to  build  more 
user  involvement.  As  users  become  interested  enough  to  post  comments  or  send  new 
ideas  to  the  repository,  other  users  will  continue  to  be  interested  to  show  up  to  see 
which  comments  have  been  added  since  the  last  time  and  possibly  find  something  of 
interest  to  their  current  situation — and  more  likely  to  find  something  applicable. 

One  way  we  have  experimented  with — to  reinforce  this  concept — is  to  list  on  the 
front  page  of  the  BPCh  tool  the  most  recently  added  practices  and  highlight  ones  that 
have  been  promoted  to  various  maturity  levels  (Bronze,  Silver,  or  Gold).  Thus,  one  of 
the  first  things  users  see  is  an  indicator  of  how  much  progress  has  occurred  since  their 
last  visit. 


CONCLUSIONS 

This  article  has  presented  some  of  the  lessons  learned  with  the  BPCh  program, 
which  aims  to  document  practices  and  quickly  disseminate  them  to  the  users.  The 
BPCh,  which  is  based  on  the  EMPEROR  approach,  makes  use  of  a  two-dimensional 
rating  scale.  These  scales  provide  users  with  a  quick  overview  of  the  trustability  and 
maturity  of  the  stored  practice  records.  The  scales  allow  users  to  understand  and  to 
draw  their  own  conclusions  based  on  a  set  of  evidence  from  different  contexts,  from 
research  studies  as  well  as  industrial  experiences,  and  using  measures  at  different 
levels  of  rigor.  Practitioners  can  rely  on  this  information  without  reading  in  detail 
through  the  different  evidence  sources,  unless  they  are  interested  in  the  very  detailed 
level  of  information. 

In  addition,  ways  to  collect  user  feedback  and  trigger  discussions  are  offered  to 
allow  a  vivid  and  growing  user  community.  While  initial  feedback  regarding  the 
BPCh  tool  has  been  positive  (Turner  &  Shull,  2005),  we  are  continuing  to  improve 
the  BPCh  program  and  its  associated  tool  through  ongoing  research,  advisory  groups, 
and  user  community  feedback.  We  are  interested  in  addressing  such  questions 
as:  “How  much  extra  effort  to  certify  evidence  sets  and  summaries  as  correct  is 
worthwhile  to  users?”  or  “Are  there  subsets  or  types  of  evidence  that  users  will  find 
especially  worthwhile?” 

We  invite  you  to  take  a  look  at  our  BPCh  tool,  available  at  http://bpch.dau.mil.  We 
appreciate  all  feedback,  whether  it  be  submitted  through  the  tool  or  directly  to  the 
authors’  e-mail. 
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SENSE  AND  RESPOND: 
AN  EMERGING  DoD 
CONCEPT  FOR 
NATIONAL  DEFENSE 

Russell  A.  Vacante 


Sense  and  respond  is  a  concept  that  is  emerging  from  the  context  of  the 
network-centric  environment.  The  relative  unfamiliarity  of  this  concept  within 
much  of  the  defense  community  suggests  that  its  meaning  and  necessity 
are  not  completely  understood.  To  help  make  sense  and  respond  less  a 
catchphrase  and  more  a  well  understood  concept,  the  text  that  follows  will 
address:  what  the  term  sense  and  respond  means,  why  it  is  important  to 
our  national  security,  and  its  relationship  and  application  to  the  logistics 
community.  The  goals  of  this  article  are  to  provide  the  reader  with  a  fundamental 
understanding  of  the  sense-and-respond  concept  and  promote  greater 
dialogue  among  a  larger  group  of  interested  parties  on  this  concept. 


Sense  and  Respond  (S&R)  is  a  business  strategy  that  is  being  incorporated  for 
military  use  in  a  network-centric  environment.  The  developer  of  S&R,  Steve 
Haeckel,  the  director  of  Strategic  Studies  at  IBM’s  Advanced  Business  Institute, 
adopted  this  system  approach  to  meet  the  competitive  challenges  of  the  marketplace 
(Menotti,  2004).  Business  in  the  information  age,  he  argues,  must  respond  quickly  as 
an  adaptive  system  to  ever-changing,  unpredictable  customer  requests.  Knowing  early 
and  managing-by-wire  are  the  cornerstones  of  this  business  strategy.  The  former  is 
getting  the  jump  on  the  competition,  while  the  latter  is  using  advanced  information 
technology  to  do  so  nearly  instantaneously.  Knowing  what  is  next  on  the  business 
horizon  and  having  the  ability  to  rapidly  adapt  a  business  strategy  based  upon 
lightning-speed  information  being  received  from  the  market  environment,  improves 
both  responsiveness  to  customers  and  business  performance  in  the  competitive 
market  place.  According  to  now-retired  Navy  Captain  Linda  Lewandowski  (2003),  it 
is  this  managerial  framework,  established  by  IBM  and  now  being  adapted  for  military 
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application,  that  is  at  the  heart  of  network-centric  application  and  theory.  According 
to  Lewandowski,  key  ideas  of  S&R  include: 

1.  Demand  is  ultimately  unpredictable,  so  success  depends  on  speed  of  pattern 
recognition  and  speed  of  response. 

2.  Organizations  are  prearranged  in  modular  business  units  that  can  negotiate  with 
one  another  over  commitments. 

3.  Networks  are  self-synchronized  via  a  common  environment  and  shared  set  of 
objectives. 

4.  The  supply  chain  is  flexible. 

5.  Information  Technology  (IT)  support  enables  data  sharing,  “knowing  earlier,” 
commitment  tracking,  and  role  reconfiguration. 

Authors  Al-Hammouro,  Liberatore,  Al-Omari,  and  Phillips  (2005)  point  out  that 
“Networked  S&R  systems  extend  human  reach  beyond  temporal  and  spatial  barriers. 
Remote  physical  environments  can  then  be  monitored,  controlled,  and  affected 
through  communication  networks.” 

The  two  business  tenets  of  S&R,  knowing  early  on  and  managing-by-wire,  can 
be  adopted  for  use  by  the  military.  Similar  to  the  commercial-business  environment, 
the  military  is  also  increasingly  confronted  with  a  fast-changing,  unpredictable, 
technologically  sophisticated  global  threat.  A  military  that  can  readily  adapt,  acquire, 


FIGURE  1.  SIMPLEST  EXAMPLE  OF  S&R  SYSTEM  (WITH  A  FEEDBACK  LOOP 
BETWEEN  SENSE  AND  RESPOND)  (GARSTKA  &  ALBERTS,  2004) 
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and  assimilate  information  ahead  of  its  adversaries  will  have  the  ability  to  control 
and  dominate  the  enemy  regardless  of  the  type  and  location  from  which  the  threat 
is  being  received.  The  network-centric  environment  of  the  military  provides  the 
technical  framework  and  resources  to  receive  information  early  on  and  to  manage 
that  information  electronically.  This  improved  networking-communication  system  can 
lead  to  better  information  sharing  and  enhanced  shared  situational  awareness  (Army 
Business  Transformation  Knowledge  Center,  2006).  It  can  also  improve  the  quality 
of  decision  making.  Therefore,  within  the  defense  community,  the  concept  of  S&R 
has  significance  beyond  the  use  of  advanced  technology.  Its  primary  functions  are 
leadership,  decision  making,  and  empowerment. 

In  a  network-centric  military  environment  that  applies  the  S&R  approach,  the 
nature  of  leadership  takes  on  a  specific  characteristic.  Leadership  does  not  effectively 
occur  in  a  hierarchical  structure,  but  rather  occurs  in  a  flatter  organizational  structure 
and  is  collaborative  in  nature.  In  networked  structures,  complex  problems  are  solved 
by  people  from  diverse  organizations  working  together  in  ad  hoc  and  sometimes 
virtual  teams.  Decision  making  is  distributed  throughout  the  organization.  There  are 
few  formalized  rules  and  procedures  regulating  behavior.  As  summarized  by  Garstka 
and  Alberts  (2004),  “Individuals  in  these  organizations  come  together  to  solve  a 
particular  problem  or  set  of  problems.  Their  interactions  are  guided  by  the  nature  of 
the  problem,  which  determines  the  makeup  of  participants  based  on  their  different 
expertise.” 

With  respect  to  the  type  of  decision-making  authority  mentioned  above,  Garstka 
and  Alberts  (2004)  note  that: 

Rather  than  relying  on  rigid  and  formalized  rules  and  regulations, 
these  organizations  will  be  characterized  by  a  much  more  dynamic 
and  evolving  rule  set.  Individuals  will  need  to  be  able  to  form 
appropriate  bonds  and  identify  with  a  variety  of  groups,  from  ad  hoc 
co-located  teams  to  virtual  teams  and  networks  of  teams.  Status  will 
necessarily  result  from  performance  as  very  few  roles  will  endure 
long  enough  to  have  institutionalized  reputations.  Importantly,  these 
new  organizational  structures  will  require  a  synchronous  perception 
of  time;  multitasking  being  the  rule  not  the  exception.  Authority 
will  be  distributed  rather  than  centralized  and  finally,  the  orientation 
toward  the  environment  will  shift  from  one  based  on  control  to  one 
based  on  adaptation. 

Furthermore,  Al-Hammouro,  et  al.  (2005)  state  that: 

[The]  S&R  control  environments  can  differ  radically  in  complexity 
and  in  applications.  Such  environments  can  range  from  simple  linear 
systems  as  in  the  case  of  a  thermostat  to  very  complex  ones,  which 
might  include  systems  of  subsystems,  as  in  the  case  of  unmanned 
autonomous  vehicles  (UAVs)  and  in  the  case  of  value-chains  in 
manufacturing.  Moreover,  some  systems  may  include  different 
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hierarchical  levels  of  complexity  abstraction.  For  example  in  UAVs, 
there  are  several  hierarchical  levels.  At  the  lowest  level  is  the  direct 
force  level.  At  this  level,  the  on-board  controller  issues  tasks,  such  as 
rotating  motors  forward  or  reverse,  based  on  feedback  information 
supplied  by  sensors  (this  represents  local  S&R).  At  the  highest  level 
of  abstraction  are  software  agents.  Software  agents  carry  out  high- 
level  tasks,  and  are  responsible  for  coordinating  multiple  UAVs  into 
task-oriented  teams,  which  can  have  impact  on  different  applications, 
e.g.,  military  transformation.  On  the  other  hand,  an  online  auction 
S&R  system  would  comprise  only  the  software-agent  level. 

Agility  is  a  prominent  feature  of  the  sense-and-respond  concept,  much  as  it  is 
for  network-centric  warfare.  By  closely  examining  the  definition  of  agility,  we 
can  quickly  come  to  understand  that  there  is  a  symbiotic  relationship  between 
agility  and  empowerment:  “Agility  provides  the  ability  to  be  effective  in  changing, 
nonlinear,  uncertain,  and  unpredictable  environments.  Agile  organizations  are  the 
result  of  an  organizational  structure,  command  and  control  approach,  concepts 
of  operation,  supporting  systems,  and  personnel  that  have  a  synergistic  mix  of 
the  right  characteristics.  The  term  agile  can  be  used  to  describe  each  component 
of  an  organization’s  capabilities  and/or  an  organization  that  can  instantiate  many 
different  mission  types”  (Garstka  &  Alberts,  2004,  p.  21).  A  2005  Rand  study  of 
network-centric  operations  provides  many  threads  of  discussion  which  suggest  that 
the  implementation  of  S&R  in  an  operational  environment  is,  to  a  high  degree, 
predicated  on  empowering  soldiers  throughout  the  brigade  with  decision-making 
authority.  The  study  notes  that  “the  [situational  awareness/situational  understanding] 
afforded  platoon  leaders  and  commanders  by  the  lower  [Tactical  Internet]  and 
Force  XXI  Battle  Command  Brigade  and  Below  (FBCB2)  [gave  them]  the  ability 
to  maneuver  their  forces  and  close  with  and  destroy  the  enemy  during  urban 
operations”  (Gonzales,  et  al.,  2005).  The  author  goes  on  to  state:  “An  important 
hypothesis  captured  by  the  NCO  CF  (conceptual  framework)  is  that  individual  and 
shared  sense-making  are  improved  by  the  quality  of  interaction  supported  by  the 
network.”  This  quotation  suggests  that  command  decisions  are  knowledge-based  and 
demand-driven,  and  “depend  on  highly  adaptive,  self-synchronizing,  and  dynamic 
physical  and  functional  processes,  employing  and  enhancing  operational  cognitive 
decision  support”  (Office  of  Force  Transformation,  2005).  The  type  of  speed  and 
recognition  required  of  S&R  needs  an  empowering  organization  structure,  a  structure 
that  is  atypical  of  decision-making  processes  within  most  Department  of  Defense 
organizations. 


THE  IMPORTANCE  OF  SENSE  AND  RESPOND 

The  reality  of  21st  century  warfare  is  that  it  can  occur  at  any  place,  anywhere,  at 
anytime,  and  take  many  forms.  The  rules,  norms,  and  expectations  of  warfare  are 
highly  unpredictable.  The  Office  of  Force  Transformation  (2003)  states  that  there  is 
an: 
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Emerging  global  security  environment  [that]  represents  a  new  set 
of  challenges  and  threats,  and  fundamentally  changes  the  rules 
of  how  America  fights  its  wars.  The  new  threats  are  broader  and 
include  global,  regional,  and  local  elements.  They  are  non-state, 
multi- dimensional,  flexible,  distributed,  information  aware,  and 
rapidly  adapt  to  U.S.  strategies  and  tactics.  They  are  unconstrained 
by  the  values  that  guide  America’s  approach  to  warfare.  Increasingly, 
these  threats  have  at  their  disposal  asymmetric,  inexpensive,  and 
competitive  methods  of  creating  large-scale  effects. 

This  new  global  security  environment  is  complex  and  highly  unpredictable. 
Threats  to  U.S.  security  may  be  rooted  in  international  organizations,  nation 
states,  rogue  states,  terror  organizations,  or  any  combination  thereof  (Office  of 
Force  Transformation,  2004a).  Our  survival  depends,  to  a  high  degree,  upon 
our  ability  to  quickly  adapt  and  appropriately  respond  to  one  or  more  of  the 
changing  challenges  discussed  above.  The  S&R  system  provides  the  technology, 
information,  organizational  structure,  and  cognitive  speed  and  effectiveness  to 
prevent  minor  threats  from  such  entities  turning  into  large-scale  destructive  action 
against  the  United  States.  Since  the  speed,  intensity,  and  nature  of  the  threat  is 
highly  unpredictable,  our  response  capability  has  to  be  extremely  agile.  This  agility 
requires  “system”  response  at  the  Joint  military  level.  “Sense  and  Respond  is  a 
transformational  network-centric  concept  that  enables  Joint  effect-based  operations 
and  provides  precise,  agile  support”  (Office  of  Force  Transformation,  2004a,  p.  3). 


TABLE  1.  DISPLAY  OF  S&R  LOGISTICS  ATTRIBUTES 


FOUNDATIONS  FOR  S&RL 

GUIDING  PRINCIPLES  FOR  JOINT 
LOGISTICS  AND  DYNAMIC 
ADAPTABLE  OPERATIONS 

Dynamic  adaptation  of  logistics  support,  using 
situation  and  commander's  intent-adapted  business 
rules,  risk  thresholds,  and  multidimensional  situation¬ 
relevant  metrics 

Emphasize  achievement  of  commander’s  intent  with 
speed  and  quality  effects 

Integration  of  operations,  intelligence,  and  logistics 
functions  and  activities 

Enable  and  sustain  effects-based  operations,  using 
reconfigurable  joint/allied/coalition/treaty  organization 
force  capabilities  as  building  blocks 

Total  situation  awareness:  dynamic  knowledge  of 
evolving  commander’s  intent,  strategic,  operational, 
and  tactical  situation,  the  global  and  local  environment, 
and  logistics  and  force  capability  status 

Provide  risk-mitigated  situation-aware  preparedness, 
readiness,  deployment,  employment,  and  sustainment 
options  to  the  commander  across  the  full  range  of 
military  operations 

Functions  and  activities  governed  by  business  rules 
that  respond  dynamically,  in  real  time,  to  total  situation 
awareness 

Replace  the  cyclic  nature  of  planning  and  execution 
with  a  continuous  application  of  total  situation 
awareness  to  current  and  future  operations 

Rule-based  risk-assessed  and  mitigated  dynamic 
adaptation  of  functions,  activities,  processes, 
organizations,  supply,  and  support 

Eliminated  process  lines,  structural  lines,  organization 
overhead,  and  constraints 
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TABLE  1.  DISPLAY  OF  S&R  LOGISTICS  ATTRIBUTES  (CONTINUED) 


GUIDING  PRINCIPLES  FOR  JOINT 
FOUNDATIONS  FOR  S&RL  LOGISTICS  AND  DYNAMIC 

ADAPTABLE  OPERATIONS 


FOUNDATIONS  FOR  S&RL 

GUIDING  PRINCIPLES  FOR  JOINT 
LOGISTICS  AND  DYNAMIC 
ADAPTABLE  OPERATIONS 

Cognitive  decision  support  that  is  knowledge-guided, 
recognizes  patterns  and  anomalies,  and  mines 
feedback  and  experience  to  adapt  operations 

Provide  deterrence  and  alter  initial  conditions  by 
developing  kinetic  and  potential  force  capabilities 
relative  to  a  potential  adversary’s  courses  of  action  to: 
deny  benefits,  impose  costs,  and  induce  restraints 

Network-centric  operations  and  warfare  as  an 
operational  concept  and  as  an  infrastructure,  including 
provisions  for  robust,  secure,  agile,  and  assured 
operations,  and  operations  when  network  resources 
are  scarce 

Support  expeditionary  force  projection  and  application 
for  all  military  operations,  emphasizing  force-to-object 
maneuver 

Coordination,  collaboration,  and  coherence  of 
operations  through  robust  interfaces  and  shared 
knowledge  across  military  domains,  in  multinational 
operations,  and  with  government,  civilian,  and  private 
organizations 

Provide  logistics  for  force  capabilities  from  the 
point  of  effect  to  the  source  of  support,  using 
autonomous,  peer-to-peer,  and  brokered  demand 
and  supply 

Management  of  strategic,  operational,  and  tactical 
envelopes  of  operations  coordinated  among  the 
operations,  intelligence,  and  logistics  domains  through 
common  understanding  of  commander's  intent  and 
the  situation 

Embed  evolutionary  development  of  doctrines,  tactics, 
techniques,  procedures,  and  organization  into  the 
adaptation  of  functions  and  activities 

Broadened  asset  visibility  across:  functions;  services; 
multinational  partners;  government,  non-government, 
and  private  agencies;  the  CONUS/OCONUS 
sustaining  base;  commercial  and  opportunistic 
sources  of  supply  and  services;  and  captured  and 
confiscated  resources 

Dynamically  adapt  current  and  future  operations  to 
respond  to  evolving  commander’s  intent  and  to  the 
known  and  anticipated  status  of  situation,  environment, 
and  force  capabilities 

Proactively  sustain  and  support  the  force,  using 
knowledge-guided  adaptation,  prediction,  and 
anticipation  to  preemptively  prepare  and  provide 
logistics  resources 

Defeat  with  modest  forces  (economy  of  power)  and 
modest  reinforcement  (demassification) 

Management  of  the  mosaic  of  logistics  resources  and 
force  capabilities  and  response  to  demand  and  events 
using  advanced  information  technologies 

Increase  speed  of  command,  accommodate  dispersed, 
distributed  forces,  manage  high  rates  of  change,  and 
respond  to  closely  coupled  events 

Dynamic  adaptive  command  and  control,  using 
business  rule-adapted,  coordinated,  synchronized 
event  processing,  data/information/knowledge 
development,  awareness,  and  sharing,  and 
collaborative  functions  and  activities 

Accommodate  the  compression  of  (and  simultaneous 
distributed  response  to)  the  strategic,  operational, 
and  tactical  levels  of  war,  including  non-contiguous, 
non-linear  operations 

Tailored,  precise  logistics  enabled  through  global 
situation  awareness  and  knowledge  of  commander's 
intent,  and  local  optimization  of  resource  allocations, 
distribution,  and  employment 

Support  precision  engagement/strike,  decisive 
coercive  operations,  adaptive  warfare,  operational/ 
tactical  agility,  and  integration  of  offensive  and 
defensive  operations 
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Table  1  is  a  display  of  S&R  logistics  attributes.  It  indicates,  among  other 
things,  that  S&R  is  a  concept  that  is  applicable  to  all  units  and  troops  engaged  in 
activity  within  the  network-centric,  joint  battle  space.  Of  equal  importance,  Table 
1  underscores  the  fact  that  logistics  is  now  an  integral  part  of  the  warfighters’ 
operational  and  tactical  planning  and  execution  (Office  of  Force  Transformation, 
2004b). 


APPLYING  SENSE  AND  RESPOND  APPLICATION 
TO  THE  LOGISTICS  COMMUNITY 

The  logistics  community  must  be  prepared  to  meet  the  full  range  of  military 
operations  with  the  speed  and  agility  that  is  expected  from  the  warfighter.  It  must 
be  able  to  distribute  “guns,  bullets,  bread  and  butter”  and  adapt  its  activities  and 
support  to  quick,  ever-changing  threat  environments.  Lewandowski  (2003)  points 
out  that  “Hierarchical,  stovepipe  logistics  chains  cannot  support  distributed  adaptive 
operations.”  The  S&R  for  the  logistician  means  that  logistics  support  has  to  be  push- 
and-pull  in  nature.  Push  requires  anticipating  where  the  forces  will  be  and  delivering 
the  right  configured  package  in  the  right  place,  at  the  right  time,  in  the  right  amount. 
It  is  more  than  just-in-time,  total  asset  visibility.  It  requires  logistics  to  function  as 
an  integral  part  of  a  system.  The  very  S&R  system  that  jump-starts  our  military  into 
action  must  be  utilized  by  the  logistician  to  support  the  commander’s  intent.  As  the 
fighting  force  becomes  lighter  and  more  agile,  the  effectiveness  and  efficiency  of 
logistics  should  follow  in  kind.  It  must  be  flexible,  agile,  and  fast.  The  logistics  tail 
and  footprint  have  to  be  significantly  reduced  in  order  to  support  and  sustain  the 
troops. 


SENSE-AND-RESPOND  LOGISTICS 


Sense-and-Respond  Logistics  (S&RL)  is  a  transformational  network-centric  concept  that  enables 
joint  effects-based  operations  and  provides  precise,  agile  support.  The  S&R  Logistics  relies  upon 
highly  adaptive,  self-synchronizing,  and  dynamic  physical  and  functional  processes.  It  predicts, 
anticipates,  and  coordinates  actions  that  provide  a  competitive  advantage  spanning  the  full  range 
of  military  operations  across  the  strategic,  operational,  and  tactical  levels  of  war.  The  SR  Logistics 
promotes  doctrinal  and  organizational  transformation,  and  supports  scalable  coherence  of 
command  and  control,  operations,  logistics,  intelligence,  surveillance,  and  reconnaissance. 

Implemented  as  a  cross-Service,  cross-organizational  capability,  S&R  Logistics  provides  an  end-to- 
end,  point  of  effect  to  source  of  support  networks  of  logistics  resources  and  capabilities.  Within  S&R 
Logistics,  every  entity,  whether  military,  government,  or  commercial,  is  both  a  potential  consumer  and 
a  potential  provider  of  logistics.  It  delivers  flexibility,  robustness,  and  scalability  for  joint  expeditionary 
warfare  through  adaptive,  responsive,  real-time,  demand  and  support  networks  within  U.S.,  allied,  and 
coalition  operations  (Office  of  Force  Transformation,  2004,  p.  5). 
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As  indicated  by  the  Office  of  Force  Transformation  (2004c),  “Sense  and  respond 
logistics  will  focus  logistics  support  towards  direct  correlation  to  total  situation 
awareness.  It  will  anticipate  and  proactively  support  future  operations,  and  predict 
future  situations.”  The  ability  to  manage-by-wire  will  help  fulfill  the  commander’s 
intent.  By  fully  integrating  logistics  with  operations  and  intelligence  assets,  logistics 
resources  can  be  better  exploited.  These  resources  will  be  based  on  the  commander’s 
intent,  and  will  reduce  risk  and  uncertainty  of  delivery  and  support  as  the  redundant 
iron  mountains  of  equipment  and  supplies  give  way  to  precisely  tailored  packages 
distributed  by  a  transportation  network  that  can  transverse  the  full  spectrum  of  the 
battle  space. 

The  S&R  Logistics  Concept  is  based  upon  the  power  of  the  network  (Office  of 
Force  Transformation,  2004a).  It  uses  networks  to  support  distributed,  adaptive 
operations  in  a  seamless  and  highly  flexible  manner.  It  is  a  transparent  logistics 
system  that  the  warfighter  can  depend  upon  and  trust  to  deliver  supplies  and 


TABLE  2.  LOGISTICS:  PRE-  AND  POST-TRANSFORMATION 


PRE-TRANSFORMATION 

POST-TRANSFORMATION 

Linear 

Nonlinear 

Chains 

Networked 

Use-Based 

Effects-Based 

Service  Stovepipes 

Cross-Service  Mutual  Support 

Functional  Stovepipes 

Cross-Enterprise 

Title  Ten-Driven 

Joint  Logistics 

Pre-Planned 

Dynamic  Continuous  Planning  and  Execution 

Poor  Ops/Log  ISR  Integration 

Net  Warrior  Ethos 

Reactive 

Anticipatory 

Parametric  Analysis-Based 

Collaborative 

Hierarchical 

Networked 

Monolithic 

Distributed,  Modular 

Poor  Scalability 

Dynamically  Scalable 

Not  Flexible 

Flexible 

Consumption-Based 

Adaptive,  Cognitive 

Mass 

Speed  of  Effect 

Attrition 

Effects-Based 

Service  Perspective 

Joint  Coherence 

Efficiency 

Effectiveness 

Highly  Optimized 

Effective 

Brittle,  Rigid  Supply  Chains 

Robust,  Flexible  Demand  Networks 
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munitions  to  project  power  and  sustain  their  mission.  Table  2  indicates  how  S&R 
Logistics  will  change  the  manner  in  which  logistics  will  function  for  the  greater  part 
of  the  21st  century.  In  the  post-transformation  environment,  stovepipes  become  a  relic 
of  the  past.  Logistics,  as  with  warfighting,  will  become  a  joint  endeavor.  Networking 
will  be  the  glue  that  will  successfully  implement  cross-Service  and  cross-enterprise 
support. 

The  lightning  speed  at  which  this  responsive,  collaborative,  anticipatory  logistics 
system  must  function  requires  a  network-centric  communications  system.  This 
system  must  be  extremely  robust  and  highly  tolerant  of  nonlinear,  nonhierarchal 
communications  for  planning  and  logistics  support  for  a  broad  range  of  missions 
anywhere  in  the  world. 


THE  FEASIBILITY  OF  IMPLEMENTING  S&R 

The  feasibility  of  implementation  of  S&R  is  closely  related  to  the  necessity  for 
implementation.  The  former  Army  Chief  of  Staff,  General  Eric  Shinseki,  recognized 
that  the  threat  to  U.S.  national  security  is  taking  on  a  radically  different  form  in 
the  21st  century.  Our  forces  have  to  be  fast,  agile,  and  highly  mobile  to  respond 
to  unpredictable  sources  of  threat,  regardless  of  their  source.  Network-centric 
technology,  and  in  particular  S&R  operations,  can  enable  such  forces.  Therefore, 
S&R  feasibility  is  closely  linked  to  force  transformation  and  associated  strategic, 
operational,  and  tactical  operations  in  a  joint  warfighting  environment. 


The  feasibility  of  implementation  of  S&R  is  tlosely 
related  to  the  netessity  for  implementation. 


The  S&R  and  the  lighter,  more  agile  force  envisioned  by  General  Shinseki  are 
designed  to  work  together.  Both  are  complex  to  implement.  The  technical  training 
associated  with  both  is  unprecedented.  Both  require  a  well  educated  workforce  that 
has  received  extensive  training  in  both  warfare  methodologies.  In  addition,  the  active 
and  reserve  components  must  be  trained  to  the  same  degree  and  in  the  same  manner. 
One  important  lesson-learned  from  our  current  military  operations  in  Iraq  is  that  the 
active  force  has  been  better  trained  to  used  advanced  technology  then  their  reserve 
component  counterparts.  The  reserve  forces  often  receive  their  first  exposure  to 
advanced  technology  shortly  after  their  arrival  into  the  theater.  The  S&R  tactics  and 
technology  promise  to  be  considerably  more  complex  than  much  of  the  advanced 
technology  currently  being  used  in  Iraq.  The  military  op-tempo  is  expected  to 
continue  well  into  the  future,  suggesting  that  active  and  reserve  forces  will  continue 
to  be  deployed  together  throughout  the  world. 
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The  demographic  and  geopolitical  backdrop  of  this  situation  is  grounded  in 
predications  that  indicate  the  global  population  will  grow  from  6  to  8  billion  by  2025, 
which  will  help  to  accelerate  international  migration  that  will,  in  turn,  contribute 
to  global  nation-state  instability.  Other  issues  are  predicted  to  contribute  to  global 
instability.  For  example,  the  current  National  Security  Strategy  Plan  and  National 
Military  Strategy  are  written  to  respond  to  the  increased  fragmentation  of  states 
along  tribal,  religious,  and  ethnic  lines,  as  well  as  the  proliferation  of  new  states, 
and  the  loose  alignment  of  regional  political  and  economic  associations,  such  as  the 
Association  of  Southeast  Asian  Nations  (ASEAN)  and  Gulf  Coordination  Council. 

If  this  trend  continues,  the  demand  on  our  military  forces  will  be  the  same  or 
greater  than  today.  To  cope  with  the  sociopolitical  challenges  of  the  21st  century,  it 
is  imperative  that  S&R  capability  be  adopted.  The  feasibility  of  implementation  of 
S&R,  however,  is  contingent  upon  ensuring  that  there  is  a  cross  leveling  of  training 
among  the  active  and  reserve  component  forces. 


CONCLUSION 

What  has  been  briefly  demonstrated  in  this  article  is  that  S&R  is  a  viable  concept 
that  needs  to  be  rapidly  adopted  in  response  to  the  diverse  military  challenges  of 
the  21st  century.  The  U.S.  military  has  to  function  as  a  system  in  order  to  properly 
implement  S&R.  The  decision  maker  must  realize  that  S&R  is  not  a  concept  to  be 
solely  implemented  at  the  point  of  the  spear.  Logistics  and  related  ilities  are  just 
as  integral  to  the  system  as  the  combat  elements.  The  feasibility  of  successfully 
implementing  S&R  is  highly  dependent  on  appropriate  education  and  training  of 
the  entire  force,  requiring  a  great  deal  of  time  and  effort  that  cannot  take  place  in 
the  battle  space  during  the  fog  of  war.  The  U.S.  capability  to  respond  decisively  to 
the  military  challenges  of  the  21st  century  requires  the  near-term  implementation 
of  S&R.  With  an  understanding  of  S&R  and  its  relationship  to  national  security, 
the  logistics  community,  and  the  necessity/feasibility  of  application,  the  defense 
community  can  come  a  step  or  two  closer  towards  developing  a  S&R  plan  for 
implementation. 
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APPLICATION  OF 
STRUCTURED  DECISION¬ 
MAKING  TOOLS  TO 
DEFENSE  ACQUISITION 


LCDR  John  R.  Gensure,  USN  (Ret.) 


The  Defense  Acquisition  System  is  heavily  dependent  upon  quality  decision 
making.  The  application  of  structured  decision-making  tools  to  Defense 
acquisition  problems  can  significantly  assist  the  decision  maker  in  the 
analysis  of  complex  decisions,  particularly  those  involving  uncertainty, 
risk,  and  multiple  objectives.  Decision  analysis  and  operations  research 
are  structured  decision-making  tools  that  can  aid  the  decision  maker  in 
avoiding  biases,  documenting  decision  methodologies,  and  making  group 
decisions.  Overall,  the  systematic  application  of  structured  decision-making 
tools  can  significantly  increase  a  decision  maker's  insight  into  the  complex 
decisions  that  are  characteristic  of  the  Defense  Acquisition  System. 


D 

t 


efense  acquisition  decisions  are  often  of  extremely  high  importance  and 
consequence,  as  the  lives  of  U.S.  Armed  Forces  members  and  the  people  they 
protect  may  depend  on  the  quality  of  those  decisions.  Decision  analysis  and 
operations  research  are  two  different  structured  decision-making  methodologies  that 
can  be  employed  to  significantly  improve  the  quality  of  decision  making  and  problem 
solving,  as  well  as  provide  the  decision  maker  with  greater  insights  into  the  decision 
at  hand.  Decision  analysis  accentuates  the  decision  maker’s  objectives,  preferences, 
and  attitudes  towards  risk  (Goodwin  &  Wright,  2004).  Operations  research 
emphasizes  system  understanding  and  the  formulation  of  a  mathematical  model  of 
the  system  (Winston,  1994).  The  nature  of  the  decision  may  indicate  whether  decision 
analysis  or  operations  research  is  most  applicable,  but  in  many  cases  the  application 
of  more  than  one  technique  may  help  the  decision  maker  view  the  problem  from 
multiple  perspectives.  The  methodologies  often  complement  one  another,  providing 
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the  decision  maker  with  significant  insight  into  the  decision  at  hand  (Modjeski, 

2004). 

UNSTRUCTURED  VERSUS  STRUCTURED  DECISION  MAKING 

Decision  makers  develop  a  personalized  set  of  decision-making  tools  and  strategies 
over  time  based  on  their  experience  and  education.  When  faced  with  a  decision, 
decision  makers  employ  a  strategy  that  they  believe  to  be  the  most  applicable 
based  on  the  situation.  Characteristics  of  the  decision,  such  as  urgency,  importance, 
consequence,  and  available  information,  all  affect  a  decision  maker’s  choice  of 
strategy.  For  common  decisions  of  low  importance  and  consequence,  decision  makers 
typically  employ  unstructured  decision-making  tools  and  methods,  called  heuristics 
(Gigerenzer,  Todd,  &  ABC,  1999).  Flemishes  may  provide  satisfactory  courses  of 
action  but  often  do  not  provide  the  optimal  course  of  action  in  a  given  decision 
(Goodwin  &  Wright,  2004).  For  some  simple  Defense  acquisition  decisions,  such 
as  the  purchasing  of  copier  paper,  the  minimal  complexity  and  low  consequence  of 
the  decision  may  not  warrant  the  time  and  effort  required  to  employ  a  structured 
decision-making  tool.  For  the  purchasing  of  copier  paper,  a  decision  maker  might 
utilize  a  heuristic  strategy  where  he/she  will  rank  the  various  attributes  of  available 
vendors  in  order  of  importance  and  choose  the  vendor  that  provides  the  highest 
value  on  the  most  important  attribute  (Goodwin  &  Wright,  2004).  Should  the  lowest 
purchase  price  be  the  most  important  attribute,  corresponding  to  the  objective  of 
minimum  cost  to  the  government,  the  decision  maker  will  choose  the  vendor  that 
provides  copier  paper  that  meets  minimum  requirements  at  the  lowest  price.  If  two 
vendors  provide  copier  paper  at  the  same  lowest  price  then  the  decision  maker  will 
choose  the  vendor  that  provides  the  most  value  on  his/her  next  most  important 
attribute,  such  as  delivery  time. 

Most  Defense  acquisition  decisions  are  significantly  more  complex  than  the 
purchasing  of  copier  paper,  and  therefore  the  use  of  unstructured  heuristics  is  not 
appropriate.  In  Defense  acquisition,  decision  makers  are  typically  faced  with  complex 
decisions  involving  multiple  objectives.  As  indicated  in  the  Federal  Acquisition 
Regulations,  Part  1.102  (2005): 

The  vision  for  the  Federal  Acquisition  System  is  to  deliver  on  a 
timely  basis  the  best  value  product  or  service  to  the  customer,  while 
maintaining  the  public’s  trust  and  fulfilling  public  policy  objectives. 
Participants  in  the  acquisition  process  should  work  together  as  a  team 
and  should  be  empowered  to  make  decisions  within  their  area  of 
responsibility. 

For  Defense  acquisition  decisions  of  high  importance  and  consequence,  a  decision 
maker  should  employ  a  compensatory,  structured  decision  strategy  to  arrive  at  an 
optimal  course  of  action  versus  an  unstructured  heuristic  strategy.  Unlike  heuristic 
strategies,  which  are  noncompensatory,  a  compensatory  strategy  requires  the  decision 


262 


maker  to  not  only  rank  the  importance  of  multiple  objectives  and  their  associate 
attributes,  but  to  make  trade-offs  between  various  attributes.  Poor  performance 
by  a  decision  option  on  one  attribute  might  be  offset  by  superior  performance  on 
several  another  attributes  (Goodwin  &  Wright,  2004).  In  the  case  of  the  copier 
paper  example,  the  decision  maker  might  choose  to  purchase  copier  paper  from  a 
more  expensive  vendor  based  on  the  vendor’s  history  of  superior  delivery  times, 
responsiveness,  and  product  quality.  Decision  analysis  and  operations  research  are 
compensatory,  structured  decision-making  tools  that  can  provide  the  decision  maker 
with  significant  insight  into  complex  defense  acquisition  decisions. 


DECISION  ANALYSIS 

Robert  T.  Clemen  (1996),  Associate  Professor  of  Decision  Sciences,  Duke 
University,  provided  the  following  summary  of  the  objectives  of  decision  analysis  and 
outlined  the  decision  analysis  process  as  shown  in  Figure  1 : 


Yes 


Recommendation: 
Implement  the  Chosen  Alternative 


FIGURE  1. 

THE  DECISION  ANALYSIS  PROCESS  (CLEMEN,  1996;  VAN  DORP,  2003) 
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I  subscribe  to  the  notion  that  the  objective  of  decision  analysis  is 
to  help  a  decision  maker  think  hard  about  the  specific  problem  at 
hand,  including  the  overall  structure  of  the  problem  as  well  as  his 
or  her  preferences  and  beliefs.  Decision  analysis  provides  both  an 
overall  paradigm  and  a  set  of  tools  with  which  a  decision  maker  can 
construct  and  analyze  a  model  of  a  decision  situation... the  purpose  of 
studying  decision-analysis  techniques  is  to  be  able  to  represent  real- 
world  problems  using  models  that  can  be  analyzed  to  gain  insight 
and  understanding.  It  is  through  that  insight  and  understanding — the 
hoped-for  result  of  the  modeling  process — that  decisions  can  be 
improved. 

Decision  analysis  commences  with  a  thorough  identification  of  the  problem  and 
then  places  heavy  emphasis  on  the  subjective  judgment  of  the  decision  maker.  The 
objectives  of  the  decision  maker  along  with  his/her  preferences  are  explored  and 
evaluated  during  the  process  of  decomposing  and  modeling  of  the  problem.  Decision 
analysis  tools,  including  the  Simple  Multi-Attribute  Rating  Technique  (SMART)  and 
multi-attribute  utility  theory,  are  utilized  to  elicit  value  and  utility  functions  from  the 
decision  maker  as  well  as  his/her  attitudes  towards  risk  (Goodwin  &  Wright,  2004). 
After  the  preferred  alternative  is  identified,  sensitivity  analysis  is  conducted.  During 
sensitivity  analysis,  the  decision  maker  investigates  the  dependencies  of  preferred 
solutions  on  the  inputs  obtained  during  the  elicitation  and  modeling  stages  of  the 
decision  analysis  process  prior  to  implementation  of  the  chosen  alternative  (Goodwin 
&  Wright,  2004).  Employment  of  the  decision  analysis  process  can  provide  Defense 
acquisition  decision  makers  with  new  insights  into  complex  procurement  decisions. 


SMART  DECISION  ANALYSIS  TOOL 

The  SMART  provides  the  decision  maker  with  a  compensatory,  structured 
analytical  process  for  evaluating  complex  decisions  that  involve  multiple  objectives 
where  uncertainty  is  not  a  factor  (Edwards,  1971).  The  SMART’S  relative  simplicity, 
speed  of  application,  and  transparency — i.e.,  easy  for  individual  and  group  decision 
makers  to  understand — make  the  tool  an  extremely  valuable  asset  to  the  decision 
maker.  When  compared  to  noncompensatory,  heuristic-based  decision  methods, 
SMART  can  provide  the  decision  maker  with  a  significantly  greater  understanding  of 
complex  Defense  acquisition  decisions  (ODPM,  2004;  Goodwin  &  Wright,  2004). 

The  first  stage  of  SMART  is  to  identify  the  decision  maker.  In  the  case  of  a  new 
Defense  weapons  system  procurement,  the  acquisition  team  members  are  the  decision 
makers.  In  the  second  stage,  the  alternative  courses  of  action  are  identified.  For  a 
simplified  weapon  system  procurement  example,  the  alternatives  may  be  limited  to 
the  procurement  of  weapon  system  1  or  weapon  system  2.  In  stage  3,  the  attributes 
that  are  relevant  to  the  decision  are  identified.  For  this  example,  the  attributes  are 
determined  to  be  cost,  development  schedule,  destructive  power,  accuracy,  and  speed 
of  employment.  A  value  tree  is  displayed  in  Figure  2  (Goodwin  &  Wright,  2004). 
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Attributes 


Purchase 

Development 

Destructive 

|  Accuracy 

Speed  of  | 

Price 

Schedule 

Power 

Employment 

FIGURE  2. 

VALUE  TREE  FOR  WEAPONS  SYSTEM  PROCUREMENT  EXAMPLE 

In  stage  4,  values  for  the  performance  of  weapon  system  1  and  weapon  system  2  on 
each  individual  attribute  are  computed.  As  all  of  the  attributes  for  the  weapon  system 
procurement  example  can  be  denoted  with  quantifiable  variables,  Table  1  provides 
the  variables  and  their  associate  value  functions  for  weapons  systems  1  and  2.  In  each 
case  the  preferred  variable  is  assigned  a  value  of  1 00  and  the  least  preferred  variable 
is  assigned  a  value  of  0  (for  problems  with  additional  alternatives,  values  between 
100  and  0  would  also  be  assigned  as  appropriate)  (Goodwin  &  Wright,  2004). 

TABLE  1. 

ATTRIBUTE  VALUES  FOR  WEAPON  SYSTEM  PROCUREMENT  EXAMPLE 


Weapon  System  #1 

Weapon  System  #2 

Attribute 

Variable 

Value 

Variable 

Value 

Purchase  Price 

$1,000,000 

0 

$750,000 

100 

Development  Schedule 

12  months 

100 

14  months 

0 

Destructive  Power 

2,000  lbs 

100 

1,750  lbs 

0 

Accuracy  (Circular  Error  Probable  (CEP)) 

100  m 

100 

150  m 

0 

Speed  of  Employment 

1  minute 

0 

30  sec 

100 

In  stage  5,  the  decision  maker  is  asked  to  determine  weights  for  each  attribute 
to  reflect  his/her  preferences  between  the  attributes.  The  SMART  (Edwards,  1971) 
model  is  a  linear  additive  model  where  the  total  value  for  each  decision  option 
(weapons  systems  1  and  2)  is  the  sum  of  the  values  assigned  to  each  individual 
attribute  for  the  option  multiplied  by  its  respective  weight,  as  shown  in  Equation  1 
(ODPM,  2004): 

n 

S,  =  ^co^.  =  CO^.J  +  C025/2  +  . . .  +  CO  nsin  Equation  1 

7=1 
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The  weights  are  determined  to  reflect  the  decision  maker’s  preferences  between 
attributes.  A  simple  procedure  would  be  to  have  the  decision  maker  rank  the  attributes 
in  order  of  preference.  Unfortunately,  a  simple  ranking  method  might  provide  too 
much  weight  to  an  attribute  that  is  important  to  the  decision  maker  but  has  little 
bearing  on  the  decision  at  hand  (Goodwin  &  Wright,  2004).  For  example,  if  the 
difference  in  length  of  development  schedule  between  the  two  alternatives  was  only 
one  week,  the  importance  of  development  schedule  on  this  specific  decision  may  be 
negligible,  but  the  importance  to  the  decision  maker  of  development  schedule  as  an 
attribute  may  be  significant.  To  avoid  such  an  issue,  the  decision  maker  is  encouraged 
to  assign  swing  weight  to  each  attribute.  Edwards  &  Barron  (1994)  referred  to  the  use 
of  the  SMART  method  with  swing  weights  as  SMARTS,  which  stands  for  SMART 
with  Swings  (ODPM,  2004). 

To  determine  the  swing  weights,  the  decision  maker  is  asked  to  rank  the  attributes 
based  on  the  swing  from  least  to  most  preferred  variable  of  each  attribute  versus 
the  swing  from  least  to  most  preferred  variable  of  the  other  attributes.  The  attribute 
with  the  lowest  importance  is  assigned  a  weight  of  0  and  the  one  with  the  highest  is 
assigned  a  weight  of  100.  The  remainders  are  assigned  intermediate  values  and  then 
all  the  values  are  normalized  (Goodwin  &  Wright,  2004).  The  results  for  the  weapon 
system  procurement  example  are  shown  in  Table  2. 

TABLE  2. 

WEIGHTS  FOR  WEAPON  SYSTEM  PROCUREMENT  EXAMPLE 


Original 

Normalized 

Swing 

Swing 

Attribute 

Swings 

Weights 

Weights 

Purchase  Price 

$250,000 

100 

40 

Development  Schedule 

2  months 

30 

12 

Destructive  Power 

250  lbs 

0 

0 

Accuracy 

50  meters 

70 

28 

Speed  of  Employment 

30  sec 

50 

20 

Total 

250 

100 

Equation  1  can  now  be  utilized  in  stage  6  to  determine  the  overall  values  for 
weapons  systems  1  and  2  portrayed  in  Table  3.  Table  3  demonstrates  how  attribute 
swing  weights  and  attribute  values  can  be  combined  using  Equation  1  to  provide 
insight  to  the  decision  maker  regarding  the  weapon  system  procurement  decision. 
Purchase  of  weapon  system  2  received  a  higher  total  value  than  that  received  by 
weapon  system  1 .  After  making  a  provisional  decision  in  step  7  to  purchase  weapon 
system  2  based  on  the  results  in  Table  3,  the  decision  maker  should  complete  step  8  of 
SMARTS.  In  step  8,  sensitivity  analysis  is  completed  to  determine  how  the  results  of 
the  analysis  might  change  based  on  changes  in  the  values  and  weights  provided  by  the 
decision  maker.  Step  8  is  very  important  (and  often  neglected)  as  the  results  obtained 
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TABLE  3.  PRODUCT  OF  VALUES  AND  WEIGHTS  FOR 
WEAPONS  SYSTEM  PROCUREMENT  EXAMPLE 


will  provide  the  decision  maker  with  an  enhanced  understanding  of  the  problem 
and  better  confidence  in  the  final  Defense  acquisition  decision  (Goodwin  &  Wright, 
2004). 


UTILITY  THEORY  DECISION  ANALYSIS  TOOL 

Although  more  complicated  than  SMARTS,  utility  theory  provides  the  decision 
maker  with  a  compensatory,  structured  analytical  process  for  evaluating  complex 
decisions  that  involve  one  or  more  objectives  where  uncertainty  and  risk  are  factors 
in  the  decision.  A  utility  function  can  be  derived  from  the  decision  maker’s  attitude 
towards  risk  and  utilized  to  provide  significant  insight  into  the  decision  at  hand 
(Goodwin  &  Wright,  2004).  According  to  the  University  of  Michigan  Decision 
Consortium  (2004): 

Utility  theory  is  an  attempt  to  infer  subjective  value,  or  utility,  from 
choices.  Utility  theory  can  be  used  in  both  decision  making  under 
risk  (where  the  probabilities  are  explicitly  given)  and  in  decision 
making  under  uncertainty  (where  the  probabilities  are  not  explicitly 
given). 

To  continue  with  the  weapon  system  procurement  example,  single  attribute  utility 
theory  can  be  utilized  to  evaluate  development  schedule  risk  for  weapons  systems  1 
and  2.  In  their  work  Theory  of  Games  and  Economic  Behavior,  John  von  Neumann 
and  Oskar  Morgenstem  initiated  both  game  theory  and  the  theory  of  choice  under 
uncertainty  (Economics,  2004).  Von  Neumann  and  Morgenstern’s  (1944)  theory  of 
utility  can  be  employed  in  the  weapons  system  procurement  example  to  elicit  a  utility 
function  for  the  decision  maker’s  attitude  towards  risk  (Goodwin  &  Wright,  2004). 
The  decision  choices,  either  the  procurement  of  weapon  system  1  or  2,  along  with 
the  probabilities  of  the  development  schedule  outcomes  based  on  the  chosen  weapon 
system,  are  represented  in  the  decision  tree  shown  in  Figure  3. 
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Development 
Probability  Schedule 

15  Months 


12  Months 


13  Months 

17  Months 


14  Months 


FIGURE  3. 

DECISION  TREE  FOR  WEAPONS  SYSTEM  PROCUREMENT  EXAMPLE 

Derivation  of  the  decision  maker’s  utility  function  for  the  possible  development 
schedules  shown  in  Figure  3  can  be  accomplished  by  presenting  the  decision 
maker  with  a  series  of  hypothetical  lotteries.  First,  the  best  outcome  (12  months)  is 
assigned  a  utility  of  1.0  (u[  12  months]  =  1.0).  Next,  the  utility  of  the  worst  outcome 
is  assigned  a  utility  of  0.0  (w[17  months]  =  0.0).  Intermediate  utilities  are  calculated 
by  conducting  an  elicitation  session  with  the  decision  maker.  For  each  intermediate 
outcome,  the  decision  maker  is  asked  to  choose  between  various  hypothetical 
lotteries  which  offer  a  specific  percentage  chance  of  achieving  the  best  outcome  and 
the  corresponding  percentage  of  achieving  the  worst  outcome.  Once  the  decision 
maker  indicates  indifference  between  a  presented  lottery  and  the  actual  outcome,  the 
outcome  is  assigned  the  utility  of  the  lottery  (Goodwin  &  Wright,  2004). 

TABLE  4. 

UTILITIES  FOR  WEAPON  SYSTEM  PROCUREMENT  EXAMPLE 

u(1 6  months)  =  0.5*u(12  months)  +  0.5*u(17  months)  =  (0.5)*(1.0)  +  (0.5)*(0.0)  =  0.5 
u(1 5  months)  =  0.7*u(12  months)  +  0.3*u(17  months)  =  (0.7)*(1.0)  +  (0.3)*(0.0)  =  0.7 
u(14  months)  =  0.8*u(12  months)  +  0.2*u(17  months)  =  (0.8)*(1.0)  +  (0.2)*(0.0)  =  0.8 
u(13  months)  =  0.9*u(12  months)  +  0.1*u(17  months)  =  (0.9)*(1.0)  +  (0.1  )*(0.0)  =  0.9 
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FIGURE  4. 

UTILITY  FUNCTION  FOR  WEAPONS  SYSTEM  PROCUREMENT  EXAMPLE 


For  example,  the  decision  maker  is  asked  to  choose  between  (a)  the  certainty 
of  a  1 6  month  development  schedule  or  (b)  engaging  in  a  lottery  where  there  is 
50  percent  chance  of  a  12-month  development  schedule  and  50  percent  chance  of 
a  17-month  development  schedule.  If  the  decision  maker  indicates  that  he/she  is 
indifferent  between  the  two  choices  then  the  utility  of  a  1 6-month  development 
schedule  is  assigned  the  utility  of  that  lottery.  The  remaining  intermediate  utilities  can 
be  determined  in  a  similar  fashion  as  shown  in  Table  4.  The  decision  maker’s  utility 
function  can  then  be  graphed  as  shown  in  Figure  4. 

The  utility  function  in  Figure  4  for  the  weapons  system  procurement  example 
has  a  concave  shape  which  is  characteristic  of  a  decision  maker  that  is  risk  averse 
(Goodwin  &  Wright,  2004).  The  utility  function  can  now  be  applied  to  the  decision 
tree  in  Figure  3  to  determine  the  expected  utility  for  each  decision  option  as  shown  in 
Equations  2  and  3  and  summarized  in  Figure  5. 

(0.2*0. 7)  +  (0.8*  1 .0)  =  0.94  Equation  2 

(0.1  *0.9)  +  (0. 4*4.0)  +  (0.5*0. 8)  =  0.49  Equation  3 

Based  on  the  expected  utilities  shown  in  Figure  5,  weapon  system  1  appears  to  be 
the  preferred  option  due  to  its  higher  expected  utility,  but  prior  to  making  a  decision, 
the  decision  maker  should  perform  sensitivity  analysis  and  consistency  checks  on 
the  provided  data.  By  varying  the  information  provided  by  the  decision  maker  in  the 
elicitation  session,  the  sensitivity  of  the  calculated  expected  utilities  for  each  option 
to  changes  in  the  supplied  data  can  be  determined  and  evaluated.  Consistency  checks 
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1.0 
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0.9 
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0.0 

14  Months 

0.8 

FIGURE  5.  DECISION  TREE  SUMMARY  FOR 
WEAPONS  SYSTEM  PROCUREMENT  EXAMPLE 


can  determine  if  the  utility  function  and  calculated  expected  utilities  accurately  reflect 
the  decision  maker’s  attitudes  toward  development  schedule  risk  (Goodwin  &  Wright, 
2004). 

As  shown  in  the  weapons  system  procurement  example,  single  attribute  utility 
theory  can  be  a  valuable  tool  for  the  decision  maker  when  faced  with  complex 
decisions  involving  uncertainty  and  risk.  Multi-attribute  utility  theory  can  be  utilized 
to  extend  single-attribute  utility  theory  to  problems  involving  multiple  attributes. 
Keeney  and  Raiffa  (1976)  proposed  the  following  approach  to  derive  multi-attribute 
utility  functions  to  allow  a  decision  maker  to  evaluate  problems  involving  risk, 
uncertainty,  and  multiple  attributes.  If  mutual  utility  independence  exists  between 
the  multiple  attributes,  the  following  three-stage  process  can  be  utilized  to  obtain  the 
multi-attribute  utility  function  (Goodwin  &  Wright,  2004): 

1 .  Obtain  the  single-attribute  utility  functions  for  each  independent  attribute. 

w(Xj,  x2)  =  (k  1*m(x1))  +  (k2*«(x2))  +  (k3*«(x1)*«(x2))  Equation  4 
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2.  By  using  Equation  4,  two  single-attribute  utility  functions  can  be  combined  into 
a  multi-attribute  utility  function  (more  than  two  single-attribute  utility  functions 
can  also  be  combined  into  a  multi-attribute  utility  function,  but  the  equations  are 
increasingly  complex).  In  Equation  4,  «(x|5  xj  is  the  multi-attribute  utility  level 
when  attribute  1  has  utility  level  x  and  attribute  2  has  utility  level  xr  The  k[  and 
k2  values  are  employed  to  weight  the  single-attribute  values  and  are  evaluated  in 
a  similar  fashion  to  the  swing  weights  under  SMARTS,  except  that  lotteries  are 
utilized.  The  decision  maker  is  asked  to  choose  between  the  following  options: 

(a)  A  certain  outcome  where  attribute  1  is  at  its  best  level  and  attribute  2  is  at  its 
worst  level,  or 

(b)  A  lottery  where  there  is  a  kt  probability  that  both  attributes  will  be  at  their 
best  levels  and  a  (1  -  kj)  probability  that  both  attributes  will  be  at  their  worst 
levels. 

The  decision  maker  is  then  asked  to  choose  between  the  following  options: 

(a)  A  certain  outcome  where  attribute  2  is  at  its  best  level  and  attribute  1  is  at  its 
worst  level,  or 

(b)  A  lottery  where  there  is  a  k2  probability  that  both  attributes  will  be  at  their 
best  levels  and  a  (1  -  k2)  probability  that  both  attributes  will  be  at  their  worst 
levels. 

Equation  5  is  then  utilized  to  calculate  k3. 

k^=  \~  k~  k2  Equation  5 

3.  Complete  consistency  checks  and  sensitivity  analysis  on  the  multi-attribute  utility 
function  obtained  in  stage  2. 

As  was  the  case  with  SMARTS,  the  application  of  single-  and  multi-attribute 
utility  theory  can  provide  the  decision  maker  with  significant  insights  into  complex 
decisions.  The  SMARTS,  due  primarily  to  its  simplicity,  can  be  an  extremely  valuable 
tool  for  employment  in  problems  which  do  not  involve  uncertainty  or  risk.  When 
uncertainty  and  risk  are  involved  in  a  decision,  as  is  often  the  case  for  Defense 
acquisition  decisions,  an  understanding  of  single-  and  multi-attribute  utility  theory 
can  also  be  a  valuable  asset  to  the  acquisition  decision  maker. 
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OPERATIONS  RESEARCH 

The  U.S.  Department  of  Labor  (2004)  defines  operations  research  as: 

Operations  research  and  management  science  are  terms  that  are 
used  interchangeably  to  describe  the  discipline  of  applying  advanced 
analytical  techniques  to  help  make  better  decisions  and  to  solve 
problems.  The  procedures  of  operations  research  have  given  effective 
assistance  during  wartime  missions,  such  as  deploying  radar, 
searching  for  submarines,  and  getting  supplies  where  they  were  most 
needed. 

Wayne  L.  Winston  (1994)  provided  a  similar  definition  of  operations  research  as  “a 
scientific  approach  to  decision  making,  which  seeks  to  determine  how  best  to  design 
and  operate  a  system,  usually  under  conditions  requiring  the  allocation  of  scarce 
resources,”  and  provided  the  seven  step  operations  research  analysis  process  shown  in 
Figure  6. 


Idc 

mtify  the  Problem  or  Opportunity 

fe 

^B 

Understand  the  System  '■  y 

m 

^B 
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Formulate  a  Mathematical  Model  || 

Verify  the  Model 

1 

|  Select  the  Best  Alternative 

Iff  Present  the  Results  of  the  Analyi 

sis  S 

y|||f:  Implement  and  Evaluate 

J 
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FIGURE  6.  THE  OPERATIONS  RESEARCH  ANALYSIS  PROCESS 
(WINSTON,  1994;  HARDIN,  2004) 


As  compared  to  the  decision  analysis  process  shown  in  Figure  1 ,  which  places  a 
heavy  emphasis  on  the  subjective  judgment  of  the  decision  maker,  the  operations 
research  analysis  process  shown  in  Figure  6  places  more  emphasis  on  understanding 
the  system,  verifying  the  models,  and  formulating  detailed  mathematical  models 
which  incorporate  risk  profiles  via  probability  distributions.  A  decision  maker’s 
subjective  judgments,  particularly  those  regarding  risk,  are  not  considered  in 
operations  research.  As  indicated  by  Dr.  Richard  Modjeski  (2004): 

Decision  makers  often  are  critical  of  Operation  Research  methods 
for  ignoring  subjective  judgments.  Personal  judgments  are  a  critical 
part  of  making  good  decisions  in  decision  theory.  Decision  makers 
often  site  Operations  Research  for  being  precisely  wrong  instead  of 
approximately  right.  This  refers  to  the  tendency  to  solve  the  wrong 
problem  with  the  right  method. 

In  some  cases,  decision  makers  may  even  reject  mathematical  models  developed 
under  operations  research  that  have  been  optimized  for  the  objectives  of  the  overall 
organization  if  the  decision  makers’  personal  preferences,  objectives,  and  attitudes 
towards  risk  do  not  completely  coincide  with  those  of  the  organization.  An  example 
may  be  an  acquisition  manager  who  chooses  a  procurement  alternative  that  is  low 
in  risk  versus  an  alternative  with  greater  risk  and  potentially  higher  benefits  to  avoid 
being  associated  with  a  possible  project  failure. 

By  viewing  a  complex  problem  from  both  decision  analysis  and  operations 
research  perspectives,  a  manager  can  gain  significant  insight  into  a  decision  as  the 
two  methodologies  for  handling  risk  complement  one  another  (Modjeski,  2004).  By 
employing  both  decision  analysis  and  operations  research,  a  risk-averse  acquisition 
manager  may  be  able  to  better  balance  his/her  tendencies  towards  rejecting  a 
new  innovative  alternative  with  significant  risk  and  the  DoD’s  goal  of  exploring 
new  opportunities  and  emerging  technologies.  Decision  analysis  may  identify 
an  acquisition  manager’s  risk  aversion  and  assist  in  developing  risk-reduction 
alternatives  (Goodwin  &  Wright,  2004);  whereas  operations  research  may  identify 
how  a  high-risk  project  fits  into  the  DoD’s  overall  military  acquisition  strategy  that 
mitigates  risk  across  numerous  research  and  development  projects  throughout  the 
Defense  Acquisition  System.  A  manager  who  understands  how  to  employ  both 
decision  analysis  and  operations  research  methodologies  in  complex  decision  making 
will  be  much  better  prepared  to  strike  a  successful  balance  between  minimizing  risk 
and  maximizing  opportunities. 


CONCLUSION 

Quality  decision  making  is  critical  to  the  success  of  the  Defense  Acquisition 
System.  The  lives  of  U.S.  Armed  Forces  members  and  those  they  protect  often 
depend  on  the  quality  of  Defense  acquisition  decisions.  When  faced  with  complex 
Defense  acquisition  decisions  of  high  importance,  decision  makers  should  employ 
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compensatory,  structured  decision-making  strategies  to  arrive  at  optimal  courses  of 
action  versus  heuristic  strategies  which  provide  only  satisfactory  solutions.  Structured 
decision-making  strategies,  such  as  decision  analysis  and  operations  research 
can  provide  the  decision  maker  with  significant  insight  into  Defense  acquisition 
decisions.  Application  of  multiple  structured  decision-making  strategies  can  provide 
even  greater  insight  by  allowing  the  decision  maker  to  view  a  decision  from  multiple 
perspectives  as  the  strategies  compliment  one  another.  A  decision-maker  who  takes 
the  time  to  become  proficient  at  applying  multiple  structured  decision-making  tools 
and  strategies  will  be  much  better  prepared  to  make  quality  Defense  acquisition 
decisions,  particularly  when  faced  with  complex  decisions  of  high  importance 
involving  uncertainty,  risk,  and  multiple  objectives. 
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THE  CHALLENGES  AND 
OPPORTUNITIES  OF 
IMPLEMENTING  HUMAN 
SYSTEMS  INTEGRATION 
INTO  THE  NAVY 
ACQUISITION  PROCESS 


James  A.  Pharmer 


Over  the  last  decade,  the  Department  of  Defense  has  placed  increased 
emphasis  on  including  considerations  of  human  capabilities  and  limitations 
into  systems  engineering  and  acquisition  processes.  The  purpose  of  this 
article  is  to  provide  an  overview  of  how  the  Navy  is  implementing  Human 
Systems  Integration  (HSI),  the  process  of  incorporating  considerations, 
characteristics,  capabilities,  and  limitations  of  human  operators  and 
maintainers  within  acquisition  decision  making  at  a  level  commensurate  with 
decisions  regarding  hardware  and  software.  More  specifically,  this  article 
will  address  some  of  the  policy  initiatives,  organizational  changes,  and 
implementation  challenges  of  incorporating  HSI  into  the  acquisition  life  cycle 
to  insure  better  total  system  performance  and  lower  total  ownership  cost. 


in  the  past,  incorporating  human  considerations  into  the  military  systems 
acquisition  process  was  often  overridden  by  the  need  to  deliver  systems  to  the 
warfighter  as  quickly  and  inexpensively  as  possible.  In  fact,  there  were  some 
major  opponents  to  the  ideas  of  bringing  human  factors  into  the  fold.  Perhaps  the 
most  outspoken  of  the  critics  was  Admiral  Hyman  Rickover,  who  characterized 
the  promulgation  of  a  human  factors  program  into  the  research,  development, 
engineering,  and  production  in  shipbuilding  as  “about  as  useful  as  teaching  your 
grandmother  how  to  suck  an  egg”  (1970).  Since  that  time,  however,  several  factors 
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have  brought  the  need  for  considerations  of  human  capabilities  and  limitations  to 
light.  Among  these  were  (a)  recommendations  from  a  then-General  Accounting 
Office  (GAO)  report,  which  cited  human  error  rates  as  a  key  (in  the  range  of  50 
percent)  factor  in  major  system  failures  (GAO,  1981),  (b)  several  very  high-publicity 
military,  industrial,  and  commercial  aircraft  accidents  involving  human  error  during 
the  1970s  and  1980s,  as  well  as  (c)  the  recognition  by  the  Department  of  Defense 
(DoD)  of  the  impact  of  manpower  costs  on  total  system  life  cycle  costs.  Coupled 
together,  these  three  factors  have  brought  the  necessity  of  considering  human  factors 
to  the  attention  of  the  defense  acquisition  community.  As  a  result,  the  consideration  of 
human  factors  early  in  the  acquisition  process  was  mandated  in  the  1991  DoD5000.2 
Instructions.  Subsequent  versions  provided  more  and  more  detail  on  the  component 
domains  that  must  be  considered  in  the  acquisition  process,  including  considerations 
of  manpower,  personnel,  human  factors  engineering,  habitability,  safety,  and  health 
hazards.  While  the  instructions  have  provided  a  springboard  for  inclusion  of  these 
factors  into  the  acquisition  process,  the  real  work  remains  to  be  done  to  seamlessly 
integrate  these  considerations  into  the  systems  engineering  framework  so  that  human 
considerations  have  an  equal  footing  with  hardware  and  software  considerations  in 
the  systems  engineering  process.  Human  systems  integration  (HSI)  is  the  process  by 
which  this  is  accomplished. 


The  keyword  in  HSI  is  “ integration whith  indudes 
integration  of  human  ronsiderations  into  the  systems 
engineering  protess  as  well  as  the  integration  of 
the  domains  within  HSI . 


The  keyword  in  HSI  is  “integration,”  which  includes  integration  of  human 
considerations  into  the  systems  engineering  process  as  well  as  the  integration  of  the 
domains  within  HSI.  The  old  adage  about  giving  a  child  a  hammer  and  everything  is 
a  nail  applies  here.  Experts  in  a  specific  domain  tend  to  view  the  solutions  to  human 
performance  issues  to  be  within  their  own  respective  domain.  For  example,  from 
the  point  of  view  of  a  training  specialist,  a  human  performance  deficit  will  likely 
be  viewed  as  a  training  problem  with  a  training  solution.  However,  from  the  point 
of  view  of  a  human  factors  engineer,  the  same  problem  may  be  viewed  as  a  human 
factors  design  problem.  Furthermore,  from  the  point  of  view  of  a  manpower  analyst, 
the  same  problem  may  be  viewed  as  a  problem  of  allocation  of  tasks  to  an  operator. 
In  reality,  human  performance  problems  may  have  a  number  of  solutions  from  each 
domain  or,  more  likely,  the  solution  may  be  a  combination  of  solutions.  As  this 
illustrates,  in  addition  to  the  need  to  integrate  with  non-HSI  acquisition  and  systems 
engineering  domains,  there  is  a  need  for  integration  between  the  HSI  domains  to 
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collaborate  effectively  towards  solutions  to  human  performance  issues  within  the 
context  of  cost  and  schedule  constraints. 

While  the  implementation  of  HSI  is  a  challenge  for  all  branches  within  DoD, 
the  purpose  of  this  article  is  to  provide  an  overview  of  the  work  that  the  Navy 
specifically  has  undertaken  on  these  challenges  with  respect  to  changes  in  policy 
and  organization,  as  well  as  the  issues  of  implementation  that  are  faced  by  major 
acquisition  programs  in  terms  of  organization,  planning,  and  conducting  analyses 


POLICY 

The  mandate  for  HSI  within  the  DoD  5000  series  brought  considerable  attention 
to  the  need  for  this  process  to  be  a  part  of  the  larger  acquisition  process  at  all  levels. 
Among  the  biggest  supporters  of  this  effort  was  Admiral  Vem  Clark,  the  Chief 
of  Naval  Operations  (CNO)  at  the  time  that  the  latest  version  of  the  5000  series 
was  promulgated.  Under  his  direction,  the  relevant  Offices  of  the  Chief  of  Naval 
Operations  (OPNAV)  and  the  major  systems  commands  for  Surface  (SYSCOMs) — 
Naval  Sea  Systems  Command  (NAVSEA),  Aviation — Naval  Air  Systems  Command 
(NAVA1R),  and  the  Space  and  Naval  Warfare  Systems  Command  (SPAWAR)  were 
tasked  to  develop  plans  to  insure  that  HSI  was  a  part  of  current  acquisition  programs. 
The  current  CNO,  Admiral  Mike  Mullen,  has  persisted  in  supporting  these  policies 
and  OPNAV  organizations  have  set  about  the  development  of  policy,  instructions, 
and  guidance  to  accommodate  the  emphasis  on  HSI  including  extensive  work  to  map 
analyses,  processes,  and  deliverables  required  for  HSI  to  align  more  readily  with 
the  systems  engineering  and  acquisition  frameworks.  This  will  insure  that  outputs 
from  the  HSI  domains  interleave  with  current  milestones  and  phases  in  a  manner 
that  allows  for  a  real  impact  on  design  decision-making  trade-offs.  Among  the  major 
Navy  initiatives  is  the  development  of  the  Systems  Engineering,  Acquisition,  and 
Personnel  Integration  (SEAPRINT)  effort,  the  goals  of  which  are  to  standardize 
Navy  HSI  policy,  ensure  HSI  issues  are  addressed,  and  to  facilitate  HSI  analyses.  The 
SEAPRINT  is  a  Naval  Enterprise-wide  approach,  which  includes  seven  actionable 
tenets  for  the  implementation.  These  are:  (a)  initiating  HSI  early  in  the  acquisition 
process,  (b)  identifying  HSI  issues  and  planning  analyses  to  mitigate  these  issues,  (c) 
insuring  that  HSI  is  “crosswalked”  throughout  relevant  acquisition  documentation, 

(d)  making  HSI  processes  a  factor  in  source  selection,  (e)  execution  of  an  integrated 
technical  process,  (f)  conducting  proactive  trade-offs  within  the  acquisition  process, 
and  (g)  conducting  HSI  milestone  assessments.  A  more  detailed  discussion  of 
SEAPRINT  is  beyond  the  scope  of  this  article.  For  more  information  on  this  initiative 
please  refer  to  the  Navy  Human  Performance  Center  website  at  https://www.spider. 
hpc  .navy.mil/index.  cffn?RID=WEB_OT_  1001399 


ORGANIZATION 

The  major  SYSCOMs  of  the  Navy,  responsible  for  developing  and  acquiring 
systems  to  support  the  warfighters,  each  had  unique  challenges  given  large 
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differences  in  organizational  structures  and  business  processes.  For  example, 
NAVSEA  was  able  to  establish  an  HSI  directorate  (NAVSEA  03)  within  their 
organization,  sanctioned  with  technical  authority  to  review  the  status  of  HSI 
performance  within  new  acquisitions,  as  well  as  to  upgrade  and  to  modify  legacy 
programs  within  an  organizational  structure  heavily  centered  on  specific  surface 
activities  and  warfare  systems. 

The  NAVAIR  organization  is  based  on  the  systems  engineering  competencies 
required  to  develop  aircraft  and  related  weapons  systems.  In  other  words,  there  is  a 
competency  for  logistics,  a  competency  for  program  management,  a  competency  for 
science  and  engineering,  etc.  Personnel  are  pulled  from  each  of  these  competencies 
directly  to  support  any  given  acquisition  program.  As  such,  responsibility  for 
specific  HSI  products  are  spread  throughout  the  organization  and  ownership  of  these 
processes  may  fall  within  several  competencies.  The  NAVAIR  approach  was  to  (a) 
institute  HSI  measures  within  its  Systems  Engineering  Technical  Review  (SETR) 
process  rather  than  develop  a  specific  directorate  charged  with  HSI  review  and,  (b)  to 
realign  specific  competencies  related  to  human  performance  science  and  technology 
to  provide  expertise  and  support  to  individual  programs  through  Integrated  Product 
Teams  (IPTs)  participation. 


The  NAVAIR  organization  is  based  on  the  systems 
engineering  t ompetendes  required  to  develop  airtraft 
and  related  weapons  systems. 


There  was  a  danger,  however,  in  having  each  of  the  Navy  SYSCOMs  developing 
unique  HSI  processes  and  organizations  in  isolation.  Without  close  coordination 
between  the  SYSCOMs,  it  was  possible,  in  fact  likely,  that  each  would  develop  its 
own  unique  ways  of  doing  business.  Given  the  complexities  of  network-centric 
warfare,  the  necessities  of  interoperability,  and  the  tight  coupling  and  integration 
between  systems  required  in  today’s  warfare,  these  unique  approaches  would  likely 
result  in  untenable  mismatches  that  would  have  serious  consequences  to  cost  and 
schedules  of  acquisition  programs,  especially  those  where  integration  of  air  and 
ship  operations  are  vital.  Consequently,  the  Navy’s  SYSCOMs  have  worked  together 
within  a  “Virtual  SYSCOM”  to  insure  that  policies  and  processes  do  not  diverge. 
Thus,  representatives  from  the  “Virtual  SYSCOM”  have  collaborated  extensively 
on  the  development  of  guidance  for  program  management  and  development  of 
metrics  for  human  performance,  technical  (i.e.,  programmatic)  performance,  and 
common  HSI  processes.  Further,  the  CNO  provided  funding  for  the  Human  Systems 
Performance  Analysis  Capability  (HS-PAC)  effort  to  develop  an  infrastructure  to 
support  the  distribution  of  data  relevant  to  human  performance,  thereby  fostering  the 
integration  of  research,  development,  and  fleet  operational  activities. 
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EDUCATION 


The  implementation  of  HSI  into  the  acquisition  and  systems  engineering  processes 
would  have  little  likelihood  of  success  without  some  effort  to  educate  the  workforce 
on  the  processes  involved.  As  such,  each  of  the  SYSCOMs  has  put  significant 
effort  toward  this  task.  Early  on,  these  efforts  focused  on  educating  a  wide  range  of 
individuals,  from  technical  directors  and  program  managers  to  systems  engineers 
to  scientists  and  engineers  within  specific  HSI  disciplines  (e.g.,  Human  Factors 
Engineering,  Manpower,  Personnel,  Training,  etc.),  on  basic  to  advanced  topics  in 
HSI.  In  part,  as  a  result  of  these  initiatives,  universities  and  colleges  are  beginning  to 
offer  degrees  and  certificates  in  HSI  as  well.  In  fact,  the  Naval  Postgraduate  School 
now  offers  a  graduate  degree  in  HSI  to  its  students. 


IMPLEMENTATION 

The  major  challenges  with  the  implementation  of  HSI  have  been  within  acquisition 
programs  themselves  because,  to  some  degree,  a  cultural  change  has  been  required 
to  more  fully  integrate  the  disciplines  of  HSI  into  well  established  systems 
engineering  and  acquisition  activities.  A  number  of  factors  have  come  into  play  on 
how  successfully  HSI  can  be  integrated  into  these  processes,  not  least  of  which  is 
acceptance  of  the  value  of  HSI  by  program  management  and  communication  of  this 
acceptance  throughout  the  program.  Other  significant  factors  include  considerations 
of  the  current  acquisition  phase  (earlier  is  better  to  be  most  effective)  as  well  as 
more  typical  concerns  of  funding  profile,  schedule  constraints,  and  trade  space  (e.g., 
legacy  equipment/new  design).  The  following  sections  will  discuss  some  of  the 
major  challenges  to  the  development  of  requirements  for  HSI,  planning  to  meet  those 
requirements  within  the  acquisition  schedule,  as  well  as  conducting  the  analyses 
necessary  to  meet  opportunities  to  insure  high  levels  of  human  performance  (and  the 
resultant  system  performance)  and  operator  situational  awareness  while  maintaining 
manageable  workload  levels. 


HUMAN  PERFORMANCE  REQUIREMENTS 

One  particularly  challenging  aspect  of  implementing  HSI  within  an  acquisition 
program  is  that  the  science  that  guides  human  performance  is  relatively  new  in 
comparison  to  the  physical  sciences,  and  much  more  subject  to  individual  variation. 
Thus,  the  development  of  requirements  and  associated  metrics  that  can  be  monitored 
and  traced  across  the  acquisition  life  cycle  is  difficult.  This  is  especially  true  for 
such  human  constructs  as  situational  awareness,  workload,  and  fatigue,  which  have 
both  physical  and  cognitive  components.  The  challenge  is  in  developing  high-level 
requirements  in  early  requirements  document  that  can  be  traced  and  monitored 
from  early  concept  exploration  and  refinement  to  sustainment  and  disposal  of  the 
system.  Workload,  for  example,  can  be  thought  of  as  the  ratio  of  tasks  to  the  time  to 
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complete  them.  However,  it  can  also  be  considered  to  include  such  cognitive  factors 
as  frustration  and  cognitive  effort.  Thus,  wide  latitude  of  interpretation  could  be  made 
with  a  requirement  to  reduce  workload  unless  tied  to  a  very  specific  and  testable 
definition  provided  in  the  requirements. 

One  approach  that  has  met  with  a  great  deal  of  success  within  the  DDG 1 000 
(formerly  DD[X])  ship  program  was  to  use  manning  levels  as  a  Key  Performance 
Parameter  (KPP)  in  the  Operational  Requirements  Document  (ORD).  In  the  case  of 
this  program,  the  threshold  value  of  this  KPP  represented  a  significant  challenge  to 
designers  that  could  most  likely  only  be  met  through  innovative  design  to  support 
fewer  operators  performing  a  much  broader  mission,  without  sacrificing  operator 
situational  awareness  or  substantially  increasing  workload.  However,  while  this 
manpower  KPP  has  worked  well  in  the  case  of  DDG  1000,  it  is  by  no  means  a 
panacea  to  be  applied  by  other  programs  as  a  substitute  for  careful  requirements  and 
function  analyses.  In  this  particular  case,  the  manpower  KPP  provided  a  useful  tool 
for  developing  metrics  to  measure  the  impacts  of  design  decisions  against  the  risk 
of  manpower  increase,  which  may  work  well  within  a  program  focused  on  a  total 
ship  but  would  probably  have  less  utility  for  smaller  craft  and/or  weapons  system. 

The  bottom  line  is  that  the  qualities  of  metrics  within  HSI  are  no  different  than  those 
usually  tracked  within  an  acquisition  programs.  They  should  be  chosen  based  on 
their  validity,  reliability,  relevance  to  the  unique  issues  of  the  mission,  and  should  be 
directly  tied  to  cost,  schedule,  and  performance  parameters  of  concern. 

THE  HSI  PRODUCT  TEAM 

Generally,  most  programs  have  integrated  HSI  by  implementing  IPTs,  or  in 
some  cases,  Cross  Product  Team  (CPT)  structures,  into  their  processes.  And,  as 
with  any  other  IPT  or  CPT  within  a  program,  success  is  dependent  on  such  factors 
as  leadership,  empowerment,  and  having  the  right  skill  mix  to  do  the  job.  Perhaps 
at  least  as  essential  to  the  success  of  these  teams  is  external  integration  with  the 
relevant  “non-HSI”  disciplines.  The  best  and  brightest  ideas  generated  within  the  IPT 
are  of  no  value  if  they  are  not  communicated  effectively  to  program  management, 
engineering,  and  design  disciplines  outside  of  the  team.  Thus,  program  management 
support  and  communication  of  that  support  throughout  the  program  are  essential  to 
the  success  of  HSI.  If  other  organizations  and  individuals  within  the  program  view 
HSI  as  purely  an  academic  exercise,  then  value  related  to  better  designed  systems  and 
decreased  total  ownership  cost  will  not  be  likely  to  be  realized. 

THE  HSI  PLAN 

Another  key  to  the  success  is  the  development  of  a  HSI  plan  that  includes  not 
only  the  goals  and  visions  for  HSI,  but  which  also  provides  a  process  and  schedule, 
which  aligns  closely  to  the  larger  program  reviews,  milestones,  and  deliverables. 
Without  this  alignment,  opportunities  for  making  a  human  performance  impact  to 
the  design  are  greatly  reduced,  being  overcome  by  acquisition  events  and  design 
decisions  already  made  without  the  benefit  of  HSI  input.  Again,  the  need  for  good 
communication  external  to  the  HSI  IPT  is  important.  Allied  with  this  issue,  however, 
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is  the  necessity  that  acquisition  programs  allow  time  and  funding  to  allow  for 
human  performance  analyses  and  trade  studies  to  be  conducted  within  sufficient 
time  to  make  these  impacts.  Thus,  while  it  is  essential  that  the  HS1  plan  align  with 
the  acquisition  schedule,  it  is  also  important  to  have  the  support  of  the  program  and 
an  understanding  that  the  dividends  of  designing  for  the  sailor  will  pay  off  in  total 
ownership  cost,  less  rework,  and  higher  total  system  (i.e.,  hardware,  software,  and 
human)  performance. 

HSI  ANALYSES 

Ultimately,  the  goal  of  HSI  is  to  integrate  considerations  of  human  capabilities 
and  limitations  into  the  design  decision-making  process  already  being  utilized  for 
hardware  and  software.  Integration  of  HSI  analysis  into  the  acquisition  and  systems 
engineering  process  is  the  key  to  achieving  this  goal.  Just  as  it  is  prudent  and 
necessary  to  perform  analyses,  testing,  and  verification  for  software  and  hardware 
integration,  these  same  activities  are  required  for  integrating  the  human  operator  into 
the  system.  The  following  discussion  describes  some  of  the  analysis  activities  that  can 
assist  in  insuring  that  this  integration  takes  place. 


Ultimately,  the  goal  of  HSI  is  to  integrate  c onsiderations 
of  human  tapabilities  and  limitations  into  the  design 
detision-making  profess  already  being  utilized 
for  hardware  and  software. 


Top-Down  Function  Analysis  (TDFA)  is  a  family  of  systematic  analyses  and 
resulting  documents  that  decompose  the  mission  of  an  emerging  system  in  a  manner 
that  links  hardware,  software,  and  human  performance  requirements  to  the  intended 
mission  (Bardine,  Goff,  &  Wilson,  2003;  Wallace,  Winters,  Dugger,  &  Lackie, 

2001;  Gordon,  Bums,  Sheehan,  Ricci,  &  Pharmer,  2005).  An  excellent  example  of 
TDFA  was  conducted  within  the  Multi-Mission  Maritime  Aircraft  (MMA)  program 
(Gordon,  Bums.  Ricci,  &  Ramsden,  2005).  The  primary  purpose  of  this  TDFA  was 
to  determine  specific  areas  on  which  to  focus  training  development.  However,  the 
data  within  this  analysis  have  value  to  support  human  engineering  decisions  on  what 
specific  tasks  and  functions  may  require  design  support  as  well  (e.g.,  decision  aids, 
display  and  control,  automation,  etc.).  Further,  these  data  could  support  decision 
making  related  to  manpower  and  the  allocation  of  tasks/functions  to  operators  and 
maintainers.  The  utility  of  such  an  approach  is  that  the  data  can  be  shared  across  HSI 
domains  to  provide  a  common  framework  from  which  to  make  engineering  decisions. 

The  DDG1000  program  has  had  a  great  deal  of  success  in  insuring  a  tight 
integration  between  the  HSI  CPT  activities  and  system  engineering  activities 
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within  the  program  through  the  Mission  System  Design  Analysis  (MSDA)  process. 
These  analyses  represent  operators  and  maintainers  within  the  context  of  mission 
performance  providing  a  means  of  expressing  functions  and  capabilities  explicitly 
across  hardware,  software,  and  human  aspects  of  the  design  (Wallace  &  McKneely, 
2006). 

Both  the  MMA  TDFA  process  and  the  DDG1000  MSDA  process  are  good 
examples  of  large-scale  Navy  programs  implementing  analysis  techniques  that 
provide  opportunities  for  dialog  between  systems  engineering  and  HSI  disciplines. 
However,  these  activities  have  had  equally  important  benefits  that  go  beyond 
strengthening  coordination  and  communication  within  their  respective  acquisition 
programs.  The  activities  have  also  provided  unique  opportunities  for  these  programs 
to  regularly  interface  directly  with  fleet  subject  matter  experts  (SMEs).  Through 
the  interaction  with  individuals  who  are  experts  in  the  military  domains,  designers 
and  developers  of  the  systems  to  support  these  domains  have  gained  a  better 
understanding  of  their  intricacies  and  complexities,  thereby  increasing  the  probability 
that  the  systems  that  are  delivered  meet  the  true  requirements  of  the  operators  and 
maintainers.  Most  HSI  professionals  would  agree  that  more  is  better  when  it  comes  to 
these  interactions. 


A  critical  tomponent  of  HSI  is  to  tondutt  human  factors 
engineering  analyses  focused  on  the  usability  of  human 
system  interfates .  Ideally,  these  attivities  are  tondutted 
iteratively  throughout  the  atquisition  life  tytle . 


As  these  programs  have  continued  to  progress  beyond  function  and  task  analysis 
activities,  the  interaction  with  the  fleet  becomes  more  and  more  important.  As  such,  a 
critical  component  of  HSI  is  to  conduct  human  factors  engineering  analyses  focused 
on  the  usability  of  human  system  interfaces.  Ideally,  these  activities  are  conducted 
iteratively  throughout  the  acquisition  life  cycle.  In  the  initial  stages  of  a  design 
concept,  human  factors  engineers  actively  participate  in  the  design  process  regularly 
conducting  such  activities  as  heuristic  evaluations  (i.e.,  usability  analysis  utilizing 
“rules  of  thumb”  of  good  human  factors  design)  and  audits  to  insure  compliance 
with  human  engineering  standards  and  guidelines.  Warfighter  feedback  is  actively 
sought  in  these  stages  through  activities  such  as  focus  groups  where  design  concepts 
are  storyboarded  and  presented  for  review  and  comment.  During  these  activities, 
SMEs  may  be  asked  to  cognitively  walk  through  scenarios  and  procedures  to  identify 
potential  usability  issues  related  to  workload,  errors,  and  situational  awareness. 

Like  the  function  analytic  activities  described  above,  heuristic  evaluation,  standards 
compliance  audits,  focus  groups,  and  cognitive  walkthroughs  have  had  the  benefits  of 
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allowing  a  better  understanding  of  the  domain,  the  opportunity  for  direct  warfighter 
involvement  in  the  design  process  and,  perhaps  most  importantly,  the  opportunity 
to  identify  potential  usability  issues  early  enough  in  the  design  process,  to  make 
an  impact  before  design  changes  are  prohibited  by  cost  and  schedule  concerns.  As 
the  design  concept  matures,  usability  testing  may  be  conducted  iteratively  by  using 
interactive  prototypes  and,  ultimately,  operational  systems  to  continue  to  identify 
usability  issues  and  to  determine  whether  the  issues  identified  represent  system  (i.e., 
hardware,  software,  and/or  human)  performance  issues  that  must  be  addressed. 

While  warfighter  interaction  is  essential  to  HSI,  programs  have  had  several 
challenges  to  taking  this  approach.  First,  active  duty  fleet  personnel  with  the 
needed  knowledge,  skills,  and  abilities  for  a  particular  domain,  especially  newly 
conceptualized  domains,  are  rarely  in  endless  supply  and  must  balance  their  time 
between  their  “day  job”  and  assisting  in  the  development  of  future  systems.  Thus, 
many  times,  there  are  simply  not  enough  qualified  participants  available  to  conduct 
test  and  assessment  able  to  conclusively  support  design  decisions.  Second,  humans- 
in-the-loop  data  are  often  difficult  to  analyze  and  interpret  in  a  timely  manner, 
especially  for  highly  complex  systems  and  warfare  domains.  Third,  a  number  of 
design  issues  focus  on  extremely  hazardous  activities  where  ethical  considerations 
would  prohibit  or  limit  the  ability  to  conduct  human  in  the  loop  evaluations. 

HUMAN  PERFORMANCE  MODELING 

Over  the  last  decade,  the  Office  of  Naval  Research  (ONR)  has  funded  a  number 
of  programs  focused  on  human  performance  modeling,  the  representation  of  certain 
aspects  of  human  behavior  in  a  form  that  allows  for  simulation-based  prediction 
(Campbell,  et  al.,  2002),  as  technologies  that  may  have  the  potential  to  mitigate  some 
of  these  issues.  One  research  thrust  of  the  ONR-sponsored  Manning  Affordability 
Initiative  (MAI)  was  to  investigate  the  potential  for  human  performance  modeling 
applications  to  design  teams  with  the  ability  to  more  rapidly  conduct  design  trade¬ 
off  activities  and  provide  opportunities  to  manipulate  projected  operator-system 
interactions  in  an  often  more  cost  effective  manner  than  humans-in-the-loop  studies. 
The  modeling  research  and  development  within  MAI  were  primarily  focused  on 
investigating  how  human  performance  modeling  could  support  human- centered 
design  to  realize  manning  reduction  on  future  naval  surface  combatants.  More 
specifically,  the  design  trade  space  utilized  for  this  research  program  centered  on 
the  development  of  a  human-centered  design  console  to  support  manning  reduction 
within  the  Air  Defense  Warfare  (ADW)  suite  of  a  combat  information  center.  Human 
performance  modeling  efforts  were  conducted  within  this  program  to  perform  design 
trade-offs  on  console  design,  the  flow  of  operator  tasks,  and  the  allocation  of  tasks  to 
operators. 

In  addition,  the  effort  investigated  techniques  for  increasing  the  fidelity  of  models 
by  verifying  model  predictions  with  data  collected  in  a  humans-in-the-loop  study 
using  current  operators  executing  a  realistic  and  challenging  ADW  scenario  on 
prototype  consoles  (Scott-Nash,  Carolan,  Humerick,  Lorenzen,  &  Pharmer,  2000). 
The  results  of  these  investigations  demonstrated  the  potential  value  of  utilizing 
human  performance  modeling  techniques  to  provide  the  engineers  with  a  structured 
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method  of  quantifying  differences  in  human  performance  between  alternative 
designs.  Human  performance  modeling  techniques  may  have  the  potential  to  help 
resolve  some  of  these  issues  and  are  being  embraced  by  the  acquisition  community. 

Examples  of  human  modeling  techniques  currently  being  utilized  by  systems 
designers  include  anthropometric,  cognitive  process,  and  task  network  modeling. 
Anthropometric  modeling  focuses  on  the  physical  attributes  of  the  projected  user,  by 
replicating  them  into  to  3-D  graphical  figures  in  order  to  provide  systems  designers 
with  realistic  ideas  on  how  conditions,  objects,  and  tasks  associated  with  the  planned 
environment  may  impact  the  human  operator.  There  are  a  number  of  cognitive 
process  models,  which,  as  their  name  implies,  function  to  simulate  some  aspect 
or  aspects  of  human  cognitive  activity.  For  example,  some  cognitive  models  have 
been  developed  to  function  as  interface  evaluation  tools  by  taking  characteristics  of 
the  task  and  interface  into  account  with  research  established  on  human  capabilities 
and  limitations  in  regard  to  perception,  cognition,  and  motor  processing  to  model 
interactions  between  the  human  operators  and  their  systems  (Campbell  et  al.,  2002). 
These  simulations  can  provide  system  designers  with  a  great  deal  of  insight  related 
to  human  interface  interaction,  thus  allowing  for  realistic  performance  predictions 
(Zachary,  Campbell,  Laughery,  &  Glenn,  1998).  Timing  and  accuracy  data  associated 
with  the  planned  work  environment  can  be  obtained  by  utilizing  task  network 
models,  which  are  discrete  event  simulations  based  on  detailed  task  requirement  data 
(Laughery  &  Corker,  1997).  If  properly  incorporated  within  the  test  and  evaluation 
phase  of  system  design,  human  performance  models  can  provide  assistance  in  both 
design  assessment  and  validation. 


CONCLUSION 

As  is  hopefully  evident  from  the  discussions  in  the  preceding  sections,  it  is 
clear  that  the  Navy  has  placed  a  great  deal  of  emphasis  on  designing  systems  with 
operators  and  maintainers  in  mind.  As  initiatives  in  organizational  structure,  policy, 
process,  and  education  continue  to  take  hold,  it  is  expected  that  the  increased 
attention  to  these  systems  will  pay  dividends  in  terms  of  better  total  system 
performance  at  a  lower  total  ownership  cost. 
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APPLYING  OFFICE 
LEADERSHIP  SOLUTIONS 
TO  MANAGEMENT 
RECOMMENDATIONS 
IN  SOLVING 

ACQUISITION  PROBLEMS 


Martin  Cain 


Leadership  is  the  process  by  which  high-performance  acquisition  takes 
place.  Influences  on  management  as  a  result  from  inspections  such  as 
internal  auditing  can  pull  attention  from  long-term  leadership  goals.  This 
article  researched  the  nonquantifiable  impacts  from  internal  auditing  and 
compared  them  to  effective  leadership  principles.  Conclusions  indicated  that 
management  could  make  better  use  of  audit  recommendations  by  applying 
leadership  to  actions  in  solving  problems  and  implementing  change. 


Management  recommendations  resulting  from  audits  or  inspections  calling  for 
corrections  in  behavior  from  leaders  and  those  they  lead  can  bring  about  a 
change  in  culture  and  attitudes  that  may  or  may  not  be  consistent  with  overall 
long-term  goals  and  objectives  of  achieving  excellence.  Solving  problems  brought 
up  in  audit  reports,  for  example,  have  the  potential  to  influence  a  manager’s  behavior 
to  act  in  a  manner  to  satisfy  the  recommendations  to  solve  problems  and  make 
improvements.  The  incentives  to  comply  with  inspector  recommendations  ought  to 
be  congruent  with  the  leader’s  desire  to  earn  the  trust  and  cooperation  from  others 
and  be  free  to  take  risks  when  the  potential  benefits  can  justify  the  actions.  In  order 
for  leaders  to  understand  and  work  within  the  organizational  culture  they  require 
an  awareness  of  the  perceptions  resulting  from  administrative  actions  that  influence 
relationships.  One  study  suggested  that  for  best  results,  leaders  should  understand 
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their  subordinates  enough  to  predict  the  outcomes  from  their  actions  and  get  the 
intended  outcomes  (Anderson,  1998).  Not  knowing  the  outcomes  from  an  audit 
inspection  could  decrease  effective  results  from  leadership  in  the  office. 


METHODS 

Consequently,  this  article  focuses  on  the  nonquantifiable  implications  from 
management  recommendations  primarily  resulting  from  audit  reports.  Data  were 
derived  from  a  research  study  exploring  leader  perceptions  to  internal  auditing,  and 
the  principles  gained  have  application  to  solving  acquisition  problems  as  well  as 
commercial  and  other  governmental  organizations  with  an  internal  audit  function. 
The  conclusions  expressed  here  relate  to  the  identification  of  leadership  outcomes 
from  the  author’s  research  study  conducted  for  a  doctoral  dissertation  for  the 
University  of  Phoenix.  The  research  included  30  interviews  with  leaders  and  an 
analysis  of  10  major  audit  reports  published  between  2000  and  2004. 


FINDINGS 

Audits  and  inspections  generate  management  movement  within  an  organization 
as  management  attempts  to  solve  problems  before,  during,  and  after  the  inspections. 
Not  only  are  the  economic  and  financial  factors  impacted,  but  nonquantifiable  factors 
relating  to  human  relations  are  also  put  into  motion.  Information  presented  here 
follows  two  principles:  (a)  Audit  influences  are  qualitative  as  well  as  quantitative,  and 
(b)  Leadership  skills  are  needed  to  address  audit  recommendations  to  management 
from  a  qualitative  perspective  of  human  relations.  Leadership  skills  are  detailed  into 
areas  of  empowerment,  management  actions,  trust  and  risk  taking,  and  potential 
leadership  benefits  from  auditing. 

An  audit  report  could  possibly  either  encourage  systems  thinking  and  long-range 
leadership  principles,  or  encourage  short-term  management  reporting  that  may 
not  be  consistent  with  the  leadership  objectives.  Influences  of  leadership  are  often 
qualitative  and  long  term.  Management  influences  are  often  quantitative  and  short 
term.  Softer  principles  of  leadership  are  difficult  to  quantify  and  include  principles 
such  as:  acting  honestly,  being  visionary,  modeling  behavior,  listening  empathically, 
displaying  kindness,  and  using  long-range  system  thinking.  Management,  on  the 
other  hand,  measures  productivity,  profits,  expenses,  and  time  in  short  periods  such 
as  quarterly  or  annually.  The  focus  on  reporting  of  short-term  results  could  detract 
from  long-range  leadership  goals  and  objectives  that  are  not  measured  as  accurately 
or  frequently  {Federal  Times,  2003b).  The  influence  on  management  from  an  internal 
audit  report  could  be  adjusted  to  help  align  this  quality  resource  to  long-term 
leadership  objectives. 
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The  value  of  an  internal  audit  may  be  distorted  if  the  value  is  solely  based  on 
quantitative  data  in  an  audit  report  and  not  the  impact  on  human  relations  within 
the  culture  of  an  organization  (Schein,  1997).  The  author’s  research  indicated 
nonquantifiable  implications  from  audit  vibrations  included  impacts  on  fear, 
stress,  tension,  motivation,  management  behavior,  and  the  perceived  quality  of  the 
workforce.  The  motivation  of  followers  and  leaders  is  impacted  by  tension  and  fatigue 
that  result  from  the  inspection  process  (Cooper  &  Sawaf,  1997).  An  understanding  of 
both  quantitative  and  nonquantitative  aspects  from  an  audit  could  help  leaders  make 
informed  decisions  and  better  understand  the  influence  of  their  actions. 


Internal  audit  reports  have  an  influeme  on 
the  actions  leaders  take  to  prepare  for  an  audit 
and  tomply  with  audit  recommendations. 


Internal  audit  reports  have  an  influence  on  the  actions  leaders  take  to  prepare  for 
an  audit  and  comply  with  audit  recommendations.  For  example,  if  internal  auditors 
identify  a  potential  to  save  $850  million,  then  management  would  be  required  to 
take  actions  to  achieve  the  predicted  results.  Responses  to  the  audit  would  require 
management  to  use  leadership  skills  to  motivate  others  to  comply  with  changes  and 
achieve  the  desired  savings.  A  lack  of  leadership  skills  in  forcing  change  could  result 
in  negative  outcomes.  The  nonquantifiable  outcomes  generated  from  management 
responses  to  an  audit  could  throw  an  organization  off  balance,  force  managers  to  be 
preoccupied  with  audit  processes,  and  distract  from  essential  leadership  objectives 
of  inspiring  others  to  change  (Strathem,  2000).  Management  actions  should  also  be 
consistent  with  efforts  to  improve  or  maintain  effectiveness  and  efficiency. 

Leadership  skills  are  needed  to  improve  the  problem-solving  effort.  In  order  to 
improve  effectiveness  and  efficiency,  management  must  respond  to  recommendations 
with  actions  that  will  not  only  improve  the  inspection  process,  but  result  in  effective 
leadership.  By  its  nature,  an  audit  tends  to  focus  on  attributes  and  factors  that  are 
quantifiable.  The  audit  process  also  serves  as  a  catalyst  that  generates  numerous 
outcomes  that  are  not  quantifiable  and  impact  team  and  individual  performance. 
Those  outcomes  may  impact  organizational  culture  and  levels  of  trust  in  leaders. 

For  example,  an  audit  resulting  in  additional  controls  over  executive  travel  expenses 
could  create  a  change  in  the  culture  of  leadership  and  perceived  level  of  trust  from 
subordinates  and  top  management.  A  change  in  management  can  come  as  a  result  of 
the  findings  and  recommendations  in  an  audit  report. 

The  findings  and  recommendations  of  audit  reports  have  an  influence  on  the 
behavior  of  management  and  the  perceptions  of  their  abilities  and  intents  to  establish 
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trust,  take  appropriate  risks,  solve  problems,  and  cope  with  change.  An  audit  often 
finds  discrepancies  and  faults  of  management,  and  the  process  tends  to  encourage 
management  to  take  corrective  actions  and  comply  with  policies  years  after  an 
audit  is  completed.  When  significant  discrepancies  are  reported  through  an  audit, 
management  can  respond  with  a  plan  of  action  to  solve  the  problem.  The  scrutinizing 
of  management  actions  can  impact  the  cultural  environment  and  influence 
perceptions  of  such  qualitative  factors  as  trust  from  subordinates  and  limitations  on 
the  extent  of  risk  taking  that  management  is  willing  to  allow.  Low  trust  can  lead  to 
low  collaboration  and  weaken  group  effectiveness  that  could  impact  the  mission  of 
the  organization  (Clegg  &  Hardy,  2002). 

To  recruit  and  retain  quality  people  managers  need  to  perform  as  leaders  in  a 
manner  that  models  the  behavior  they  seek.  The  leadership  culture  requires  attention 
to  various  aspects  of  leadership.  A  survey  conducted  by  Kunich  and  Lester  (2003) 
reported  the  number  one  concern  about  leadership  was  ineffective  communication. 
Another  big  concern  was  inappropriate  goal  setting  (p.  108).  Effective 
communication  should  include  empathic  listening  to  understand  one  another  and 
improve  group  cohesiveness.  Inappropriate  goal  setting  may  involve  goals  derived 
from  audit  actions  that  could  be  misaligned  with  overall  long-term  objectives.  The 
organization’s  structure,  processes,  and  systems  should  be  aligned  with  the  mission 
and  not  compete  with  it  or  dominate  it  (Block,  1995). 

COMMON  PERCEPTIONS  OF  AUDIT  OUTCOMES 

An  understanding  of  the  perceptions  of  audit  outcomes  could  provide  insight 
into  how  well  management  models  the  behavior  they  seek,  how  well  they  listen  to 
subordinates,  how  well  subordinates  communicate  their  perceptions  to  management, 
and  how  well  the  goals  and  actions  from  auditing  are  aligned  with  organizational 
objectives.  Many  influences  compete  for  the  attention  of  leadership.  Finding  the  right 
formula  for  successful  leadership  is  a  constant  challenge.  In  short,  understanding  the 
perceptions  of  auditing  outcomes  could  aid  in  the  development  and  growth  of  people. 
Managers  can  more  effectively  utilize  the  audit  function  through  their  leadership 
approach  to  empowering  subordinates,  managing  actions  in  resolving  audit  issues, 
and  building  trust  while  encouraging  risk  taking.  Each  of  these  approaches  could  lead 
to  potential  leadership  benefits  from  auditing. 

The  perceptions  from  auditing  could  impact  the  way  managers  empower  others 
and  promote  trust  and  appropriate  risk  taking.  A  major  part  of  leadership  is  “growing 
people”  by  empowering  others  and  giving  them  training  and  learning  opportunities 
to  broaden  their  perspectives  and  abilities  (Weber,  1996).  As  people  are  developed 
through  empowerment,  the  culture  of  trust  is  impacted.  Trust  and  appropriate  risk 
taking  are  part  of  the  qualitative  culture  of  leadership.  Subordinates  need  to  trust 
the  leaders  to  be  fair  and  have  the  best  interest  of  the  organization  and  society  as  a 
priority.  Of  necessity,  leaders  and  empowered  subordinates  need  to  be  encouraged 
to  take  appropriate  risk  in  being  innovative  in  finding  solutions  to  tough  problems 
and  encouraging  the  same  in  others.  Leaders  will  have  a  greater  insight  into  how  to 
add  value  to  people  when  they  understand  the  thoughts  and  feelings  of  the  people 
they  serve.  A  lack  of  success  could  have  serious  consequences  in  accomplishing  the 
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mission  of  the  organization.  Since  leaders  need  to  comply  with  laws,  the  findings  in 
audit  reports  identifying  leaders  either  complying  or  not  complying  with  laws  and 
rules  could  impact  the  image  of  leadership  by  the  subordinates  and  consequently 
impact  trust  (Kunich  &  Lester,  2003).  Research  into  leadership  perceptions  of 
auditing  could  impact  organizational  performance  through  understanding  of  how  well 
trust  is  developed. 

Since  management  of  materials  and  processes  in  an  audit  can  impact  the  leading 
of  people,  management  actions  taken  as  a  result  of  an  audit  could  influence  the 
perception  of  leadership  (Bennis,  2002).  Attitudes  and  perceptions  of  leadership 
impact  retention,  morale,  productivity,  and  dedication,  which  are  all  essential  for  a 
healthy  organization  (Katz  &  Kahn,  1978).  Michael  Quinn  Patton  (2002)  provided 
an  example  of  the  need  to  focus  on  more  than  just  the  quantifiable  outcomes  of  an 
audit.  An  audit  may  place  “too  much  emphasis  on  things  that  can  be  quantified  so 
that  it  misses  the  results  .  .  .  that  are  not  easily  measured”  (p.  18).  Examples  of  those 
difficult-to-measure  outcomes  in  Patton’s  example  were  anxiety,  low  trust,  and  an 
undesirable  atmosphere  at  work. 

RESPONSES  TO  AUDIT  RECOMMENDATIONS 

In  responding  to  audit  recommendations,  management  could  be  responsible  for 
actions  to  improve  fiscal  economies,  managerial  controls,  and  logistical  outcomes. 
These  managerial  actions  are  designed  to  provide  better  policies,  structures,  and 
processes  to  more  efficiently  utilize  resources  to  achieve  effective  outcomes  (Drucker, 
1999).  Actions  involving  saving  money  or  strengthening  controls  for  better  long- 
range  outcomes  also  impact  the  people  that  carry  out  those  actions.  The  statements 
and  actions  of  the  managers  simultaneously  affect  the  image  and  identity  of  the 
organization  (Hatch  &  Schultz,  1997).  Audit  actions  then  in  theory  could  contribute 
to  the  identity  and  image  of  the  organization  that  could  impact  perceptions  that 
influence  trust. 

A  lack  of  trust  could  be  motivated  by  leader  managers  not  modeling  the  behavior 
they  desire  in  others  or  giving  directions  that  are  inconsistent.  Major  motivators  of 
management  actions  are  internal  audit  reports  that  not  only  report  on  the  effectiveness 
of  management’s  actions,  but  also  give  recommendations  and  record  actions  taken,  or 
to  be  taken,  by  management  to  improve  operations.  The  perceptions  of  subordinates 
on  those  actions  may  be  a  hidden  factor  that  is  not  part  of  the  decision  criteria. 

A  lack  of  appropriate  risk  taking  could  be  related  to  management  actions  of  not 
empowering  subordinates  or  requiring  compliance  to  internal  controls  that  may  be 
tied  to  short-term  quantifiable  measures  rather  than  long-term  or  qualitative  factors. 
Again,  this  condition  is  related  to  leadership  and  could  also  be  interrelated  with 
action  involving  an  audit.  Semler  (1995)  reported  that  he  removed  many  policy 
manuals  from  his  company  (Semco)  and  relied  on  his  managers  to  make  good 
decisions  that  impacted  their  stewardship.  This  nonquantifiable  control  of  trust 
resulted  in  a  better  control  than  the  measured  controls  that  existed  in  policy  manuals 
designed  to  maintain  order.  The  overall  long-term  outcome  of  the  nonquantifiable 
controls  resulted  in  survival  and  profits  for  the  company  when  competitors  were 
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losing  profits.  Empowering  subordinates  is  just  one  management  resource  that  could 
be  impacted  by  an  audit  report. 

The  culture  can  be  manifested  in  the  roles  people  play.  Under  the  leader-role 
theory  described  by  Bass  (1990),  “Leaders  behave  according  to  what  is  expected 
of  them”  (p.  44).  If  the  image  of  the  organization  defines  the  leader’s  role  as  one 
that  takes  immediate  and  decisive  action  to  solve  a  problem,  the  actions  help  form 
attitudes  and  perceptions.  The  same  principle  could  be  applied  to  followers  if  they  in 
turn  would  behave  according  to  what  is  expected  of  them.  Thus  the  transformational 
role  of  leaders  may  be  acted  out  without  conscience  direction  of  the  implications 
from  the  roles  played  (Couto,  1995).  The  role  of  an  authoritarian  leader  could 
produce  one  result,  and  the  role  of  democratic  leader  could  produce  another.  Those 
results  are  a  manifestation  of  the  changes  initiated  by  the  leaders  using  the  culture  of 
the  organization. 

Understanding,  learning,  and  morale  are  all  essential  ingredients  for  improved 
leadership.  Leaders  cannot  get  followers  to  understand  them,  until  the  leaders 
understand  the  followers.  As  Kunich  and  Lester  (2003)  put  it,  “Leaders  need  to  know 
and  understand  their  subordinates”  (p.  42). 

IDENTIFYING  THE  AUDIT  FUNCTION 

The  audit  function  can  either  focus  on  policing  efforts  for  management  or 
emphasize  learning  and  consulting.  Although  an  audit  may  conduct  studies  to 
assess  compliance  with  laws,  policies,  and  regulations,  an  audit  can  also  act  as 
feedback  to  improve  effectiveness  of  operations.  Auditing  should  support  every 
level  of  management  with  middle  managers  the  prime  customers  in  a  stewardship 
environment  to  give  high-performance  potential  to  human  resource  policies  (Block, 
1995). 

The  morale  and  emotional  well  being  of  subordinates  determine  the  effectiveness 
of  operations  and  should  be  a  major  emphasis  of  the  audit  function  (Block,  1995, 
p.  147).  When  top  management  is  the  primary  customer  of  auditing,  it  gives  the 
appearance  that  auditors  act  as  the  eyes,  ears,  and  voice  of  top  management  and 
creates  a  “separation  between  those  who  do  the  work  and  those  that  manage  the 
work”  (Block,  p.  117).  The  Government  Accountability  Office  and  its  standards  are 
an  example  of  “policing  in  the  name  of  help”  (Block,  p.  119). 

Audits  stimulate  change.  Kanter  (2000)  recognized  the  need  for  involvement  to 
get  people  to  change.  Leadership  is  required  to  create  the  involvement  to  lower  the 
resistance  to  change.  Wren  (1995)  noticed  that  resistance  builds  when  trying  to  force 
people  to  analyze  assumptions  in  their  work.  Actions  forced  on  people  from  auditing 
can  also  create  an  equal  amount  of  resistance  or  compliance  depending  on  the 
participation  and  involvement  of  those  required  to  change.  Chris  Argyris  was  quoted 
by  Bass  (1990)  as  saying,  “An  organization  will  be  most  effective  when  its  leadership 
provides  the  means  whereby  followers  may  make  a  creative  contribution  to  it  as  a 
natural  outgrowth  of  their  needs  for  growth,  self-expression,  and  maturity”  (p.  43). 
Lollowers  of  auditing  outcomes  may  also  make  creative  contributions  to  resolve  audit 
findings  and  be  enthused  about  the  improvement  process.  Participative  leadership 
suggests  leaders  create  the  conditions  for  members  of  the  group  to  feel  free  to 
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actively  solve  problems  (Bass,  1990,  p.  437).  Understanding  and  effective  leadership 
can  overcome  much  resistance  to  change  and  the  related  cultural  values. 

The  strategies  used  to  fix  problems  in  the  present  should  not  be  structured  to  set 
off  a  chain  of  events  that  will  require  more  attention  and  resources  in  the  future 
(Oliver,  2002).  Cause  and  effect  are  separated  by  time  and  space  and  the  effect  of 
changes  may  not  happen  for  a  long  time  after  the  changes  are  made.  Senge  (1994) 
cautioned  against  short-term  fixes  which  only  appear  to  make  problems  go  away.  The 
effects  of  some  actions  may  not  appear  for  years,  giving  management  the  impression 
that  the  short-term  fix  was  effective.  Watered-down  compromises  that  reflect  murky 
assumptions  could  be  full  of  contradictions  that  decrease  trust  and  support  from 
subordinates  who  are  left  to  face  the  effects  of  management  actions. 

THE  KEY  ROLE  OF  LEADERSHIP 

Leadership  is  the  process  by  which  effective  management  takes  place.  For 
example,  Senge  (1994)  warned  that  the  process  of  emphasizing  financial  accounting 
as  the  only  system  to  deal  with  neglects  the  dynamic  complexity  of  the  conditions 
that  create  the  accounting  reports.  By  the  same  principles,  an  audit  that  focuses  only 
on  financial  or  economic  systems  may  neglect  the  long-term  impact  from  the  soft 
systems  that  manage  the  accounting  systems. 


Leadership  is  the  protess  by  whith 
effective  management  takes  plate . 


The  human  side  of  systems  thinking  increases  understanding  of  the  humans 
involved  and  the  corporate  culture  in  which  they  operate.  The  culture,  attitudes, 
and  assumptions  of  the  followers  will  impact  the  way  they  perceive  the  leaders.  The 
integrity,  discipline,  and  desires  of  the  followers  will  determine  what  they  consider 
important.  If  the  goal  of  a  worker  is  to  gain  material  goods  through  the  easiest  route, 
the  behavior  may  reflect  the  attitude.  If  the  goal  of  the  worker  is  personal  mastery  and 
growth,  the  behavior  could  be  significantly  different. 

The  soft  systems  may  be  hard  to  quantify.  Senge  (1994)  wrote,  “No  one  will  ever 
be  able  to  measure  to  three  decimal  places  how  much  personal  mastery  contributes 
to  productivity  and  the  bottom  line”  (p.  146).  Fie  defines  personal  mastery  as  the 
discipline  to  clarify  personal  vision  and  as  seeing  reality  objectively  (p.  7). 

Likewise,  the  nonquantifiable  outcomes  from  leadership  may  not  be  tied  directly 
to  profits,  but  all  outcomes  have  an  impact.  Actions  that  take  away  from  subordinates’ 
desires  for  personal  mastery  and  growth  could  be  detrimental  to  exceptional 
productivity.  Actions  that  transform  people  into  better  people  are  results  of  positive 
leadership  outcomes.  Sensitive  leaders  need  to  cultivate  the  true  dedication  and 
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innovation  from  subordinates  who  are  part  of  the  complex  soft  systems  involving 
organizations  and  the  nonquantifiable  leadership  perceptions  and  outcomes  from 
auditing. 


RECOMMENDATIONS 

The  research  exploration  of  nonquantifiable  leadership  perceptions  supports  the 
following  recommendations  for  management  to  either  do  or  continue  to  do  as  they 
strive  for  excellence  and  high  performance. 

1 .  Managers  closer  to  audit  findings  should  resolve  the  issues  using  knowledge  of 
the  culture  and  audit  perceptions  to  guide  their  decisions. 

2.  Management  responses  to  auditing  should  take  the  emotional  well  being  of 
their  people  into  consideration  when  giving  directions  that  result  from  audit 
recommendations. 

3.  Management  should  raise  awareness  of  the  perceptions  of  the  policing  aspect  of 
their  actions  and  adjust  actions  to  compensate  for  those  perceptions. 

4.  Organizational  management  should  be  aware  of  problem-solving  outcomes  in 
order  to  be  a  step  closer  in  the  progress  to  a  high-performance  organization  rather 
than  a  high-compliance  organization. 

5.  The  internal  audit  focus  should  expand  to  serve  management  with  suggestions  on 
how  their  leadership  could  be  improved. 

6.  Internal  auditing  should  expand  to  include  recommendations  based  on  human 
resources  and  relationships  rather  than  economics  to  improve  the  success  rate  of 
implementing  changes. 

7.  Auditing  and  management  should  address  the  leadership  methods  through  an 
open  system  recognizing  multiple  influences,  including  the  soft  systems  of 
human  interrelations. 

8.  Management  should  bring  in  a  systems-thinking  approach  when  responding  to  an 
audit  to  include  the  soft  systems  of  the  human  activities  (Checkland,  2001). 

While  implications  from  the  above  recommendations  would  be  positive, 
resources  would  still  be  required  to  expand  the  use  of  auditing  and  apply  additional 
understanding  and  knowledge  to  making  changes.  The  investment  in  this  effort 
should  be  worth  the  effort  in  the  long  term  as  the  organization  runs  more  efficiently 
and  effectively  to  perform  its  mission. 
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SUMMARY 

The  exploration  of  the  perceptions  of  leaders  influenced  by  auditing  indicated 
management  concepts  of  structure  and  control  were  emphasized  at  the  expense  of 
leadership  concepts  of  human  relations.  The  management  functions  of  auditing 
and  strengthening  internal  controls  were  separated  from  the  leadership  role  of 
understanding  human  emotions  and  effectively  motivating  followers  to  make  changes. 
While  auditing  is  a  management  tool,  the  activity  should  be  combined  with  leadership 
roles  to  fully  benefit  from  auditing.  A  leadership  focus  in  the  audit  process  as  a  cause 
of  either  positive  or  negative  findings  could  be  a  method  of  integrating  management 
with  the  leadership  role  and  becoming  more  effective  at  both  managing  and  leading. 
Also,  a  leadership  emphasis  in  taking  management  action  and  implementing 
changes  to  solve  problems  could  make  the  effort  more  effective  in  leading  people 
and  creating  the  conditions  for  them  to  be  productive.  People  and  organizations  win 
when  leadership  skills  are  applied  to  management  actions  on  audit  recommendations. 
Effective  problem  solving  in  acquisition  will  only  take  place  through  an  effective 
leadership  process. 
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The  introduction,  which  should  not  be  labeled,  opens  the  body  of  the  paper  and 
states  the  problem  being  studied  and  the  rationale  for  the  research  undertaken. 

The  methods  section  should  include  a  detailed  methodology  that  clearly  describes 
work  performed.  Although  it  is  appropriate  to  refer  to  previous  publications  in  this 
section,  the  author  should  provide  enough  information  so  that  the  experienced  reader 
need  not  read  earlier  works  to  gain  an  understanding  of  the  methodology. 

The  results  section  should  concisely  summarize  findings  of  the  research  and  follow 
the  train  of  thought  established  in  the  methods  section.  This  section  should  not  refer 
to  previous  publications,  but  should  be  devoted  solely  to  the  current  findings  of  the 
author. 

The  discussion  section  should  emphasize  the  major  findings  of  the  study  and  its 
significance.  Information  presented  in  the  aforementioned  sections  should  not  be 
repeated. 


RESEARCH  CONSIDERATIONS 

Contributors  should  also  consider  the  following  questions  in  reviewing  their 
research-based  articles  prior  to  submission: 

■  Is  the  research  question  significant? 

■  Are  research  instruments  reliable  and  valid? 

■  Are  outcomes  measured  in  a  way  clearly  related  to  the  variables  under  study? 

■  Does  the  research  design  fully  and  unambiguously  test  the  hypothesis? 

■  Are  needed  controls  built  into  the  study? 

Contributors  of  research-based  submissions  are  also  reminded  they  should  share 
any  materials  and  methodologies  necessary  to  verify  their  conclusions. 


CRITERIA  FOR  TUTORIALS 

Tutorials  should  provide  special  instruction  or  knowledge  relevant  to  an  area  of 
defense  acquisition  to  be  of  benefit  to  the  DoD  Acquisition,  Technology,  and  Logistics 
workforce. 
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Topics  for  submission  should  rely  on  or  be  derived  from  observation  or 
experiment,  rather  than  theory.  The  submission  should  provide  knowledge  in  a 
particular  area  for  a  particular  purpose. 


OPINION  CRITERIA 

Opinion  articles  should  reflect  judgments  based  on  the  special  knowledge  of  the 
expert  and  should  be  based  on  observable  phenomena  and  presented  in  a  factual 
manner;  that  is,  submissions  should  imply  detachment.  The  observation  and  judgment 
should  not  reflect  the  author’s  personal  feelings  or  thoughts.  Nevertheless,  an  opinion 
piece  should  clearly  express  a  fresh  point  of  view,  rather  than  negatively  criticize  the 
view  of  another  previous  author. 


MANUSCRIPT  STYLE 

We  will  require  you  to  recast  your  last  version  of  the  manuscript,  especially 
citations  (endnotes  instead  of  footnotes),  into  the  format  required  in  two  specific  style 
manuals.  The  ARJ  follows  the  author  (date)  form  of  citation.  We  expect  you  to  use  the 
Publication  Manual  of  the  American  Psychological  Association  (5th  Edition),  and  the 
Chicago  Manual  of  Style  (15th  Edition). 

Contributors  are  encouraged  to  seek  the  advice  of  a  reference  librarian  in 
completing  citations  of  government  documents  because  standard  formulas  of  citations 
may  provide  incomplete  information  in  reference  to  government  works.  Helpful 
guidance  is  also  available  in  Gamer,  D.  L.  and  Smith,  D.  H.,  1993,  The  Complete 
Guide  to  Citing  Government  Documents:  A  Manual  for  Writers  and  Librarians  (Rev. 
Ed.),  Bethesda,  MD:  Congressional  Information  Service,  Inc. 


COPYRIGHT  INFORMATION 

The  ARJ  is  a  publication  of  the  United  States  Government  and  as  such  is  not 
copyrighted.  Because  the  ARJ  is  posted  as  a  complete  document  on  the  DAU  home 
page,  we  will  not  accept  copyrighted  articles  that  require  special  posting  requirements 
or  restrictions.  If  we  do  publish  your  copyrighted  article,  we  will  print  only  the  usual 
caveats.  The  work  of  federal  employees  undertaken  as  part  of  their  official  duties  is 
not  subject  to  copyright  except  in  rare  cases. 

In  citing  the  work  of  others,  it  is  the  contributor’s  responsibility  to  obtain 
permission  from  a  copyright  holder  if  the  proposed  use  exceeds  the  fair  use 
provisions  of  the  law  (see  U.S.  Government  Printing  Office,  1994,  Circular  92: 
Copyright  Law  of  the  United  States  of  America,  p.  15,  Washington,  DC:  Author). 
Contributors  will  be  required  to  submit  a  copy  of  the  written  permission  to  the 
Managing  Editor  before  publication. 
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COPYRIGHT  POLICY 

We  reserve  the  right  to  decline  any  article  that  falls  into  these  problem  copyright 
categories: 

■  The  author  cannot  obtain  official  permission  to  use  previously  copyrighted 
material  in  the  article. 

■  The  author  will  not  allow  DAU  to  post  the  article  with  the  rest  of  the  ARJ  issue  on 
our  home  page. 

■  The  author  requires  that  unusual  copyright  notices  be  posted  with  the  article. 

■  To  publish  the  article  requires  copyright  payment  by  the  DAU  Press. 


MANUSCRIPT  FORMAT 

Pages  should  be  double-spaced  and  organized  in  the  following  order:  title  page, 
abstract,  body,  reference  list,  author’s  note  (if  any),  and  figures  or  tables.  Figures  or 
tables  should  not  be  inserted  (or  embedded,  etc.)  into  the  text,  but  segregated  (one 
to  a  page)  following  the  text.  If  material  is  submitted  on  a  computer  diskette  or  e- 
mailed,  each  figure  or  table  should  be  saved  to  a  separate,  exportable  file  (i.e.,  a 
readable  EPS  file).  For  additional  information  on  the  preparation  of  figures  or  tables, 
see  CBE  Scientific  Illustration  Committee,  1988,  Illustrating  Science:  Standards 
for  Publication,  Bethesda,  MD:  Council  of  Biology  Editors,  Inc.  Please  restructure 
briefing  charts  and  slides  to  a  look  similar  to  those  in  previous  issues  of  the  ARJ. 

The  author  (or  corresponding  author  in  cases  of  multiple  authorship)  should 
attach  to  the  manuscript  a  signed  cover  letter  that  provides  all  of  the  authors’  names, 
mailing  and  email  addresses,  telephone  and  fax  numbers.  The  letter  should  verify 
that  the  submission  is  an  original  product  of  the  author;  that  it  has  not  been  published 
before;  and  that  it  is  not  under  consideration  by  another  publication.  Details  about 
the  manuscript  should  also  be  included  in  this  letter:  for  example,  title,  word  length, 
a  description  of  the  computer  application  programs,  and  file  names  used  on  enclosed 
diskettes  or  in  email  attachments,  etc. 


AUTHOR  PHOTOS 

Please  send  us  a  cover  letter;  biographical  sketch  for  each  author  (not  to  exceed  70 
words);  head  and  shoulder  print(s)  or  digitized  photo(s)  (saved  at  300  pixels  per  inch, 
at  least  5X7  inches,  and  as  a  TIFF  or  JPEG  file);  prints  of  photos  will  be  accepted 
and  returned  upon  request;  one  copy  of  the  printed  manuscript;  and  any  diskettes. 
These  items  should  be  sturdily  packaged  and  mailed  to:  Department  of  Defense, 
Defense  Acquisition  University,  Attn:  DAU  Press  {Defense  ARJ  Managing  Editor), 
Suite  3,  9820  Belvoir  Road,  Fort  Belvoir,  VA  22060-5565. 
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DEFENSE  ARJ  PRINT  SCHEDULE 


2007-2008 


The  Defense  ARJ  is  published  in  quarterly  theme  editions.  Please  consult  the  DAU 
home  page  for  current  themes  being  solicited.  See  print  schedule  below. 


Publication  Date 

February  2008 
April  2008 
August  2008 
December  2008 


Due  Date 


1  August  2007 
1  October  2007 
1  February  2008 
1  June  2008 


In  most  cases,  the  author  will  be  notified  that  the  submission  has  been  received 
within  48  hours  of  its  arrival.  Following  an  initial  review,  submissions  will  be  referred 
to  referees  and  for  subsequent  consideration  by  the  Executive  Editor,  Defense  ARJ. 

Contributors  may  direct  their  questions  to  the  Managing  Editor,  Defense  ARJ,  at 
the  address  shown  above,  or  by  calling  703-805-3801  (fax:  703-805-2917),  or  via  the 
Internet  at  norene.fagan-blanch@dau.mil. 

The  DALI  Flome  Page  can  be  accessed  at:  http://www.dau.mil. 
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Statement  Required  by 
the  Act  of  August  1 2,  1 970 
Section  3685,  Title  39,  U.S.C. 
Showing  Ownership,  Management, 
and  Circulation 


The  Defense  Acquisition  Review  Journal  (ARJ)  is  published  at  the  Defense  Acquisition 
University  (DAU),  Fort  Belvoir,  VA.  The  University  publishes  four  issues  annually.  The 
Director  of  the  Defense  Acquisition  University  (DAU)  Press  is  Eduard  Boyd,  the  Managing 
Editor  of  the  Defense  ARJ  is  Norene  Taylor,  and  the  publisher  is  the  DAU  Press.  All  are 
collocated  at  the  following  address: 

DEFENSE  ACQUISITION  UNIVERSITY 
ATTN  DAU  PRESS  (Defense  ARJ) 

9820  BELVOIR  ROAD,  SUITE  3 
FORT  BELVOIR  VA  22060-5565 


AVERAGE  NUMBER  OF  COPIES  EACH  ISSUE  DURING  THE  PRECEEDING  1 2  MONTHS 


A.  Total  number  of  copies  printed  (net  press  run) . 1 8,292 

B.  Paid  and/or  requested  circulation: 

1 .  Sales  through  dealers  and  carriers,  street  vendors,  and  counter  sales . 0 

2.  Mail  subscriptions  paid  and/or  requested . 1  7,804 

C.  Total  paid  and/or  requested  circulation . 1  7,804 

D.  Free  distribution  by  mail,  carrier,  or  other  means;  samples, 

complementary,  and  other  free  copies . 436 

E.  Total  distribution . 18,240 

F.  Copies  not  distributed: 

1 .  Office  use,  leftover,  unaccounted,  spoiled  after  printing . 52 

2.  Returns  from  news  agents . 0 

C.  Total . 18,292 


ACTUAL  NUMBER  OF  COPIES  OF  SINGLE  ISSUE  PUBLISHED  NEAREST  TO  FILING  DATE 


A.  Total  number  of  copies  printed  (net  press  run) . 1 8,344 

B.  Paid  and/or  requested  circulation: 

1 .  Sales  through  dealers  and  carriers,  street  vendors,  and  counter  sales . 0 

2.  Mail  subscriptions  paid  and/or  requested . 1  7,896 

C.  Total  paid  and/or  requested  circulation . 1  7,896 

D.  Free  distribution  by  mail,  carrier,  or  other  means;  samples, 

complementary,  and  other  free  copies . 398 

E.  Total  distribution . 18,294 

F.  Copies  not  distributed: 

1 .  Office  use,  leftover,  unaccounted,  spoiled  after  printing . 50 

2.  Returns  from  news  agents . 0 

C.  Total . 18,344 
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First 
initial 
of  last 
name 


Defense  ARJ  Defense  AT&L 

QUANTITY _  QUANTITY _ 

NEW  SUBSCRIPTION 
CANCELLATION 
CHANGE  OF  ADDRESS 


DATE  FORM  COMPLETED 

LAST  NAME  _ 

FIRST  NAME  _ 

NEW  ADDRESS _ 

OLD  ADDRESS  _ 

DAY/WORK  PHONE  _ 

SIGNATURE  (Required)  . 


The  Privacy  Act  and  Freedom  of  Information  Act 

If  you  provide  us  your  business  address,  you  may  become  part 
of  mailing  lists  we  are  required  to  provide  to  other  agencies 
who  request  the  lists  as  public  information. 

If  you  prefer  not  to  be  part  of  these  lists,  use  your  home  address. 
Please  do  not  include  your  rank,  grade,  service,  or  other  personal 
identifiers.  In  accordance  with  U.S.  Postal  Service  regulations, 
electronic  mail  or  faxed  subscriptions  are  not  permitted. 
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ATTN:  DAU  PRESS 
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